A few years ago I took over webmaster duties for a very active community forum. About a month into the job the unthinkable happened, we lost the table that contained all of the posts!
I had backups (7 day rolling system) but the backups were a good 24 hours old and the server kept dropping lines while it was chugging away at restoring the database table.
I needed to go into the sql file and pull out all of the lines for the post table but I had one very large problem; The sql file was 524MB and my computer only had 128MB of RAM!
I tried to cajole my computer into opening the file but it was simply too big. Rather then curse my poor under-powered computer I got an idea.
Since I knew the structure of the sql commands I could write a program to open the file, line-by-line, and take the lines I needed and put them in a new file.
Now, it is important to note that I was unaware of the program Grep at the time. If I had known of Grep back then I would have just used that!
So, using some very simple C programming language functions I built a program that would search any text file for a given search string and output it all to a new file. Even though the post table itself was still huge, the shear number of sub-forums allowed me to break it down into files for each sub-forum.
Now I had about 124 sql files, one for each area of the forums. It was a simple process to load each one up onto the server and load the table data back into the mySQL server. Then I had only to a do a quick check of the number of items in the table compared to the number of lines in the file to be sure all the statements were executed.
I give the following code to you to use, modify, etcâ€¦ I ask only that you follow my Creative Commons Attribution-ShareAlike 2.5 License.
It would be a very easy modification to change this program to search an entire directory of text files for a given string. I imagine some of those â€œspy on your kidsâ€ program do just thatâ€¦
Please note that this was thrown together in about 5 minutes and is not optimized nor does it have any safegaurds written into it.
using namespace std;
const int MAX_LEN = 102400 ;
int main(int argc, char *argv)
int i = 0;
// give info on how to run program
// Creative Commons Attribution-ShareAlike 2.5 License
printf("sqlquery copyright 2005 Stephen De Chellis n");
printf("sqlquery takes 2 argumentsnn");
printf("sqlquery [filename] [search string]nn");
printf("[filename] = name of file to open for searchingn");
printf("[search string] = string to search the file fornn");
printf("sqlquery will search the file line by line and store the resultsn");
printf("in a new file named [search string].sqln");
i = 0;
infile.open (filename, ifstream::in);
printf("unable to find file: %s",filename);
outfile.open (filename2, ofstream::out);
printf("unable to create file: %s",filename2);
outfile << line << endl;
printf("Found %i matches",i);