[Bug] [Crash][Reproducible] `EOFError: Ran out of input` when import yapf with multiprocess

[whlook]

previous similar issue #1164

Problems

• When i use yapf (just import it) in multiprocess program, it will crash with 'EOFError: Ran out of inpu'，log info below:

Reproduction

1. yapf==0.40.2
2. rm -r ~/.cache/YAPF
3. run this python code:

 import multiprocessing
 
 def proc():
     import yapf
 
 if __name__ == "__main__":
     pp=[]
     for i in range(100):
         p = multiprocessing.Process(target=proc)
         pp.append(p)
 
     for p in pp:
         p.start()
 
     for p in pp:
         p.join()
                                                   

Reason

the problem code is here:https://github.com/google/yapf/blob/c0908d043b45f082e6339cd767c28e6d697cb22e/third_party/yapf_third_party/_ylib2to3/pgen2/driver.py#L247

When in multi process enviroment,some process will create grammar cache(when ~/.cache/YAPF not exists), before the process write to the cache file, other process will see the file exist and load it, but the cache file is empty right now , so EOFError will raise and the program crashed.

How to fix it?

• I think we need to add try here(https://github.com/google/yapf/blob/c0908d043b45f082e6339cd767c28e6d697cb22e/third_party/yapf_third_party/_ylib2to3/pgen2/driver.py#L247) like we had add try when we create the cache.
• Or we can add a multiprocess.Lock here
• Or we can do python -c 'import yapf' before we start our multiprocess program.(for users,trick)