A few years ago I took over webmaster duties for a very active community forum. About a month into the job the unthinkable happened, we lost the table that contained all of the posts!
I had backups (7 day rolling system) but the backups were a good 24 hours old and the server kept dropping lines while it was chugging away at restoring the database table.
I needed to go into the sql file and pull out all of the lines for the post table but I had one very large problem; The sql file was 524MB and my computer only had 128MB of RAM!
I tried to cajole my computer into opening the file but it was simply too big. Rather then curse my poor under-powered computer I got an idea.
Since I knew the structure of the sql commands I could write a program to open the file, line-by-line, and take the lines I needed and put them in a new file.
Now, it is important to note that I was unaware of the program Grep at the time. If I had known of Grep back then I would have just used that!
So, using some very simple C programming language functions I built a program that would search any text file for a given search string and output it all to a new file. Even though the post table itself was still huge, the shear number of sub-forums allowed me to break it down into files for each sub-forum.
Now I had about 124 sql files, one for each area of the forums. It was a simple process to load each one up onto the server and load the table data back into the mySQL server. Then I had only to a do a quick check of the number of items in the table compared to the number of lines in the file to be sure all the statements were executed.
I give the following code to you to use, modify, etc… I ask only that you follow my Creative Commons Attribution-ShareAlike 2.5 License.
It would be a very easy modification to change this program to search an entire directory of text files for a given string. I imagine some of those “spy on your kids†program do just that…
Please note that this was thrown together in about 5 minutes and is not optimized nor does it have any safegaurds written into it.
#include <fstream>
#include <iostream>
#include <string.h>
#include <stdio.h>
using namespace std;
const int MAX_LEN = 102400 ;
int main(int argc, char *argv[])
{
int i = 0;
if(argc!=3)
{
// give info on how to run program
// Creative Commons Attribution-ShareAlike 2.5 License
printf("sqlquery copyright 2005 Stephen De Chellis n");
printf("sqlquery takes 2 argumentsnn");
printf("sqlquery [filename] [search string]nn");
printf("[filename] = name of file to open for searchingn");
printf("[search string] = string to search the file fornn");
printf("sqlquery will search the file line by line and store the resultsn");
printf("in a new file named [search string].sqln");
return 0;
}
char line[MAX_LEN];
char filename[256];
char filename2[256];
i = 0;
strcpy(filename2,argv[2]);
strcat(filename2,".sql");
strcpy(filename,argv[1]);
ifstream infile;
ofstream outfile;
infile.open (filename, ifstream::in);
if(!infile)
{
printf("unable to find file: %s",filename);
return 0;
}
outfile.open (filename2, ofstream::out);
if(!outfile)
{
printf("unable to create file: %s",filename2);
return 0;
}
printf("n working");
do
{
infile.getline(line,MAX_LEN);
if(strstr(line,argv[2])!=NULL)
{
outfile << line << endl;
i++;
printf(".");
strcpy(line,"");
}
}while(infile.eof()==0);
printf("Found %i matches",i);
infile.close();
outfile.close();
return i;
}