Is it ok to delete the old updates when the new update is added?
Which files/folders are you actually referring to ? The one under AppData/local/Infinity path and directory (app-2.0.xx folder)?
I think it wouldn’t pose any problem, if you delete previous old update files, imo, though, I recommend you make a BACKUP of any file/folder which you are trying to delete.
Just to be on a safer side, as sometimes deleting files may prevent the app from Launching properly. Btw, the folder shouldn’t take a lot of HDD/SSD space as well…