openFrameworks icon indicating copy to clipboard operation
openFrameworks copied to clipboard

ofSplitString is slow!

Open ofTheo opened this issue 3 years ago • 0 comments

In some cases the difference can between ofSplitString and this implementation can be 200:1 in terms of speed.

Using a different implementation ( that only works for single chars as seperators )


//--------------------------------------------------------------
static vector<string> splitString(const string &s, char delim) {
    stringstream ss(s);
    string item;
    vector<string> tokens;
    while (getline(ss, item, delim)) {
        tokens.push_back(item);
    }
    return tokens;
}

The test ( csv.txt is just random comma separated values ):

    float t1 = ofGetElapsedTimef();
    auto arr = ofSplitString( ofBufferFromFile("csv.txt"), ",");
    float t2 = ofGetElapsedTimef();
    
    float t3 = ofGetElapsedTimef();
    auto arr2 = splitString( ofBufferFromFile("csv.txt"), ',');
    float t4 = ofGetElapsedTimef();
    
    cout << " arr size " << arr.size() << " arr2 size " << arr2.size()  << endl;
    cout << " time for OF " << t2 - t1 << " time for splitString " << t4-t3 << " how much faster? " << (t2-t1)/(t4-t3) << endl;
    
 arr size 978 arr2 size 978
 time for OF 1.48444 time for splitString 0.00714064 how much faster? 207.887

For someone trying to load a large textile the difference was 10 minutes vs 7 seconds. Thinking we can probably speed up the current ofSplitString without loosing functionality.

ofTheo avatar Aug 23 '22 22:08 ofTheo