Fast search and replace in large files with sed

Last week i had to search and replace all occurences of a string inside a relatively big MySQL database dump file. My previous experiences with search and replace actions in files of similiar size or bigger suggested that this was going to take me a while. Normally i would write a small PHP script to do the search and replace action for me. However, recently i’ve been looking to find better, more productive, ways to do everyday things. So after a quick google i found the *nix tool sed.

What sed is, is best described from it’s manual:

Sed is a stream editor. A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline). While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed’s ability to filter text in a pipeline which particularly distinguishes it from other types of editors.

The command to search and replace is similar to the syntax you would use in vi.
Let’s say you have a file database.sql and want to replace every occurence of myolddomain.com to mynewdomain.com. You would use the following sed command:

$ sed -i 's/myolddomain.com/mynewdomain.com/g' database.sql

By executing this command sed will go through your file, searching and replacing every occurence of myolddomain.com within moments. In my case, on files of 18MB and 32MB, the search and replace took under a second. But will take just moments on files much bigger than that.
Since sed is a command line tool and accepts all kinds of input, be it streamed or piped, it is a tool that is suited for a lot of different use cases.

Sed is probably good for a lot more than just search and replace. But i’ll have to look more into it to write something meaningful about it. If you want to read more about sed or check out the slides of the presentation “Sed & Awk – The dynamic duo” by Joshua Thijssen.

Oh by the way, as i mentioned sed is a linux/unix tool, but also seems to be available for Windows. It’s probably included in Mac OS distributions as well.

  • Adsf

    Thanks. So good running on a mac. Running a fancy os and still have all the powerful unix libs. Thanks for sharing ! :)

  • I’ve tried most softwares like Geany, Dreamweaver, BlueFish, Notepad++ but none of this worked. Will try this, hope this works..
    Thanks

  • Steven

    Thanks! replaced 20,000 instances in a second! Worth noting I had to add extra ” to make work like so:
    $ sed -i ” ‘s/myolddomain.com/mynewdomain.com/g’ database.sql