duplicate entry error in unified log

[jdalbey]

In the unified log below, note after the error loading json that the CAD log entries are repeated (with a timestamp of 0:00). It's as though the lastLineNum global variable in cad_watcher.py was reset to zero somehow.

0:01:16, CAD log, Incident #206, sam: SEND OUT FOR PIZZA
0:01:31, CAD log, Incident #202, sam: NOW IS THE TIME
0:04:56, CMS Activated.,N I-5 25.82 Yale OC, 'LET'S GO FLY:A KITE.::::'
0:05:46, CAD log, Incident #206, sam: SEND OUT FOR PIZZA
0:05:46, CAD log, Incident #202, sam: NOW IS THE TIME
0:05:46, CAD log, Incident #206, sam: FOR ALL GOOD MEN
0:05:52, CAD log, Incident #202, sam: TO COME TO THE AID OF THEIR COUNTRY.
0:08:27, CMS Deactivated., N I-5 25.82 Yale OC
0:08:52, CAD log, Incident #202, sam: WE SHOVEL SAUSAGE
Error loading json for elapsed time
0:00:00, CAD log, Incident #206, sam: SEND OUT FOR PIZZA
0:00:00, CAD log, Incident #202, sam: NOW IS THE TIME
0:00:00, CAD log, Incident #206, sam: FOR ALL GOOD MEN
0:00:00, CAD log, Incident #202, sam: TO COME TO THE AID OF THEIR COUNTRY.
0:00:00, CAD log, Incident #202, sam: WE SHOVEL SAUSAGE
0:37:34, CAD log, Incident #202, sam: NOW IS THE TIME again

So far I've been unable to reproduce this error.

[jdalbey]

Here's another example, showing it's not related to json error. It just happened randomly at 13:10 when nothing else was going on.

0:02:09, CMS Updated., N I-405  0.87 IRVINE CENTER DRIVE, 'USE ALT ROUTE:::::'
0:03:04, CAD log, Incident #187, Sam: HELLO WORLD
0:03:24, CAD log, Incident #210, Sam: EAT MORE GREENS
0:04:04, CAD log, Incident #205, Sam: NOW IS THE TIME
0:05:14, CMS Updated., N I-405  0.87 IRVINE CENTER DRIVE, 'USE 5 NORTH:::::'
0:07:09, CAD log, Incident #187, Sam: SEND DONUTS
0:13:10, CAD log, Incident #187, Sam: HELLO WORLD
0:13:10, CAD log, Incident #210, Sam: EAT MORE GREENS
0:13:10, CAD log, Incident #205, Sam: NOW IS THE TIME
0:13:10, CAD log, Incident #187, Sam: SEND DONUTS

[jdalbey]

Unable to reproduce, but I'm guessing that it occurs when cad_watcher tries to read the comments log at the exact time CADserver is writing the file. I enhanced cad_watcher with more elaborate error handling to check for missing file and empty file. Assume an empty file is a read sync issue and don't do anything, whereas a missing file means a "reset" and start the log over. Committed in r467.