Getting a Random String From a Big File with PHP

By | May 2, 2008

When you’re working with big files, it is not ok to use such functions like file(), file_get_contents, or anything else that loads the whole file content into memory. You often need to take just one random string from a very big file. Let’s take we have a file with proxy servers and should select one of them for further usage. Here is the solution:

//Let’s Open the file
$proxyfile=fopen (“./proxies.txt”, “r”);

fseek($proxyfile, 0, SEEK_END);
// Let’s get its filesize (for big files filesize() does not work)
$len = ftell($proxyfile);
// Taking a random start position (minus 20 is added to prevent choosing a random vaue in the last string)
$posrand=mt_rand(0, $len)-20;
// Moving to the random position of the file
fseek($proxyfile, $posrand);
// getting the line starting from random position (it may be incomplete, because the random position may point to the middle of a string)
$line = fgets($proxyfile);
// Taking a proxy from the next string that should be complete
$proxy= trim(fgets($proxyfile));

This works with files of any size, especially when you need to select an absolutely random string.