Improve Program Performance

I have a javascript program which executes operations with my JSON data (200Mega) This programm search an occurences in my datas by regxp var myDatas = [ { 'id' : '0000000001',

Solution 1:

If the issue is that the UI is frozen during processing, you can break up the processing into chunks that are processed asynchronously. This will let the UI respond to the user.

var i = 0,
    chunkSize = 10, // This is how many to process at a time
    duration = 30;  // This is the duration used for the setTimeout

(functionprocess() {
    // process this chunkvar chunk = myDatas.slice(i, i += chunkSize);

    // do the next chunkif (i < myDatas.length)
        setTimeout(process, duration);
})();

You can tweak the chunkSize and duration as needed.

If the issue is to improve actual code performance, you need to show some code.

Solution 2:

The answer posted by user1689607 is interesting, but using setTimout will not execute the code asynchronously. It will be run on the same thread as the main application. What's still interesting in using a timer is that the other parts of the application will have some processing time available while timer is running.

Best way to have your application working ok on slow/fast devices is to choose a processing time and a 'release' time. You also need, since processing data might take some time, to be able to cancel it. And use a callback to handle the end of process.

The code could look something like that :

var DataProcessor = {
     dataProcessDuration : 30,
     releaseDuration : 5,
     dataProcessedCallback : function() { 
                //... somehow notify the application the result is available
                                       },
     _currentIndex : 0,
     _processing : false,
     _shouldCancel : false,
     processData : function() 
                              {   this.cancelProcessing();
                                  this._currentIndex =0;
                                  this._processing = true;
                                  // launch data process now, or give it time to cancelif (!this._shouldCancel)  this._processData();
                                  else setTimeout(this._processData.bind(this), 
                                            this.releaseDuration+this.dataProcessDuration+1);
                               },
     cancelProcessing : function() 
                               { 
                                     if (!this._processing) return;
                                     this._shouldCancel=true;
                               },
     _processData : function() {
          if (this._shouldCancel) { this._shouldCancel = false ;  
                                    this._processing = false; 
                                    return; }
           var startTime = Date.now();
           // while there's data and processing time  left.while  ( ( this._currentIndex < length of data)
                       && (Date.now()-startTime < this.dataProcessDuration) )
                {    var thisBatchCount=10;
                     while (thisBatchCount--)
                           {  
                              // ... process the this._currentIndex part of the datathis._currentIndex++;
                             if ( this._currentIndex == length of data) break;
                            }
                }
           if (this._currentIndex == (length of data) ) // the end ?
                {     this._processing = false; 
                      this.dataProcessedCallback()
                 }
           else// keep on processing after a release time
                setTimeout(this._processData.bind(this), this.releaseDuration);
                           }
}

Rq : in the main process loop you might want to fine-tune the granularity by changing thisBatchCount. If too small you waste time by over-checking Date.now(), and too big the inner processing will take too much time and you won't have the process/release ratio you have choosen.