js.readFileSync read empty (JSON.parse err) while watching json file modified from python

Question

I use node.js to write a server.js file, and it will watch the json file modified by python every second.

If the file was modified, the server will read this json file and print it out. It will work. But the error of JSON.parse: SyntaxError: Unexpected end of JSON input happens sometimes.

Here's my file generation code with python.

di = {"lon":str(0),"lat":str(0)}
iteration = 0
while True:
    time.sleep(5)
    lon = np.random.randint(-180, high=180)
    lat = np.random.randint(-90, high=90)
    di["lon"] = str(lon)
    di["lat"] = str(lat)
    with open('./generate.json', 'w') as f:
        json.dump(di, f, ensure_ascii=False)
    
    # with open(f'./{iteration}.json', 'w') as f:
    #    json.dump(di, f)
    print(f"Done. iteration: {iteration}, lon: {lon}, lat: {lat}")
    iteration += 1

The watching function of my server.js

generate_file_path = './navigation/generate.json'

let fsWait = false;
fs.watch(generate_file_path, function(event, filename){
    if (filename && event==='change'){
      if (fsWait) return;
      fsWait = setTimeout(()=>{
        fsWait = false;
      }, 100);
      let rawdata = fs.readFileSync(generate_file_path, 'utf8');
      try{
        let g_file = JSON.parse(rawdata);
        console.log(g_file);
      }catch (e){
        console.log(e);
      }
    }
});

And the error will like this. enter image description here

I found that it might be a readfile problem. I changed my server code like this.

let fsWait = false;
fs.watch(generate_file_path, function(event, filename){
  if (filename && event==='change'){
    if (fsWait) return;
    fsWait = setTimeout(()=>{
      fsWait = false;
    }, 100);
    let rawdata = fs.readFileSync(generate_file_path, 'utf8');
    try{
      // let g_file = JSON.parse(rawdata);
      console.log(rawdata);
    }catch (e){
      console.log(e);
    }
  }
});

And it will sometimes print nothing.

The right hand side is python generation log, left hand side is server.js. (iteration 7, 10, 17 was missing that read by server.js)

I've tried modified the sleep time of python but the error of reading empty file still exists. Since the fs.readFileSync sometimes works and sometimes fails, I don't know how to debug this kind of issue.

Haven't yet. I stuck at the fs.readFileSync problem. It will sometimes read an empty file just like the post above. — domokun0413
– domokun0413, Commented Dec 14, 2020 at 3:00

Sam Hasler · Accepted Answer · 2021-06-18 16:11:31Z

0

The watch triggers multiple times. That's why you need the setTimeout. Just move the reading into the setTimeout so that it doesn't get caught by the other events.

let fsWait = false;
fs.watch(generate_file_path, function(event, filename){
  if (filename && event==='change'){
    if (fsWait) return;
    fsWait = setTimeout(()=>{
      fsWait = false;
      let rawdata = fs.readFileSync(generate_file_path, 'utf8');
      try{
        // let g_file = JSON.parse(rawdata);
        console.log(rawdata);
      }catch (e){
        console.log(e);
      }


    }, 100);
  }
});

answered Jun 18, 2021 at 16:11

Sam Hasler

12.7k10 gold badges74 silver badges106 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

js.readFileSync read empty (JSON.parse err) while watching json file modified from python

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related