It’s no secret that apart from programming I enjoy a good relaxing game to take my mind off things. This is the sole reason I still have Windows OS installed in a partition. Recently with Steam embracing Linux more and more my Archlinux machine also allows me to play some nice games.
This post will address syncing saves for various games that don’t support Steam Cloud, but focus solely on Pillars Of Eternity as an example.
Syncing saves on the cloud
The easy way
Steam offers its games the ability to synchronize saves on the cloud across multiple machines and platforms. The developers of the game have to take advantage of the feature though. One such very nicely working example is Terraria.
All you need to do is make sure that the cloud icon under a character or a world is activated and then you are set.
Before I played around with Steam in Linux I was playing this game in Windows. When I installed Steam in ArchLinux and got Terraria, my character and the world I created was there ready for me to play. Really awesome.
The not so easy way
Unfortunately some games don’t utilize the Steam cloud sync in a cross platform way and can’t offer the same service that Terraria does. One such example is Pillars of Eternity as can also be seen by posts like this. True enough I encountered the same problem when trying to sync my saves from Windows to Linux. Steam cloud was just not working.
To make matters worse I also got 2 Linux machines which I would also like to keep in sync. Thankfully the PoE save games from Windows are indeed compatible with Linux as I have tested this myself.
A kind of brute-force way to sync them would be to use Dropbox. The save games location differs depending on your Operating system so check it here. Simply copy the whole directory to Dropbox and then copy from Dropbox to the target machine when you want to sync. Unfortunately this can be a hassle and hard to maintain.
The proposed tool
I really like Pillars Of Eternity and would appreciate to be able to sync saves between different machines without a hassle. This is why I spent a bit of time to write a tool to do that.
The tool is a python script and is in Github. It has been written with a more Generic purpose in mind for syncing saves to Dropbox for any game that does not support it. At the moment only Pillars of Eternity is tested and even for that DO NOT USE without backing up your saves directory first. You have been warned.
Calling the script is pretty straighforward:
This script assumes that you also have dunst installed. If you don’t or if you don’t want to get desktop notifications for each change the script does then also add the
--no-notify option to the script.
Once called the script will attempt to check all the save files from the local location and see if a corresponding save file is in Dropbox. If yes then it will check the save timestamp to see which one should be kept and keep it. Finally all Dropbox saves will also be checked to see if any saves are missing from the local directory.
Script in Depth
As I mentioned above this was written with generic saves in mind but for the moment the only game for which the sync process is a bit trustworthy is Pillars of Eternity. The reason for that will be made clear soon.
Embedded in the script for now (later could, and probably should be a different file) is a list of
GameEntry objects which should be filled in by the user and explain which games to sync, how, and from which directories.
The GameEntry class is:
def defaultSaveNameCB(f): return basename(f) def defaultGetSaveTime(f): return getmtime(f) class GameEntry(): def __init__( self, name, steamPath, dropboxPath, saveSuffix=None, saveNameCB=None, saveTimeCB=None ): """ A Game Entry. It consists of: name: A name for the game steamPath: Path where steam stores the saves for this game dropboxPath: Dropbox path where you need the saves backedup. Will be created if it does not exist. saveSuffix: Optional suffix for the save game files. If it is not None then only files with the given suffix in the directory will be considered. saveNameCB: Optional callback to determine the name of the save file to compare by checking the filename or contents. If the function returns \"__IGNORE__\" then that save file is ignored. if the function returns the empty string there has been an error saveTimeCB: Optional callback to determine the in-game time the save was performed. If not given then the last file modification timestamp is taken which can't really be very trustworthy. """ self.name = name self.dropboxPath = dropboxPath self.steamPath = steamPath self.saveSuffix = saveSuffix self.saveNameCB = saveNameCB if not saveNameCB: self.saveNameCB = defaultSaveNameCB self.getSaveTime = saveTimeCB if not saveTimeCB: self.getSaveTime = defaultGetSaveTime
A GameEntry is basically a desciption of a game and how its saves should be handled. It is composed of:
- name: A name to identify the game with
- dropboxPath: A path to the location to which the saves should be synced to and synced from. I recommend using Dropbox path for this purpose.
- steamPath: The path where the Game places its save files which we want to sync.
- saveSuffix: An optional suffix for the game files. If given, then any files in the directory not ending with this suffix will be ignored.
- saveNameCB: A function to identify a name for the save game from the file itself. If none is given then the filename is used as the save name identifier.
- saveTimeCB: A function to identify a save time from a save file. This is a really important function since it’s what determines which of 2 save files is newer and should be kept. The default function checks the file’s last modified date.
Pillars Of Eternity GameEntry
The tool can have many entries but comes with a premade entry for Pillars Of Eternity (PoE). This is how it looks like for my machine.
gamesList = [ GameEntry( "PillarsOfEternity", "/home/lefteris/.local/share/PillarsOfEternity/SavedGames", "/home/lefteris/Dropbox/saves/PillarsOfEternity", "savegame", POESaveName, POESaveTime ) ]
A nice TODO, would be to add different locations for different OSes as this now only works for Linux.
If you check inside a save file for PoE you will actually notice that it’s nothing more than a zip archive.
There is one particular file of importance inside that archive which we are interested in and that is saveinfo.xml. Example of the contents of such a file are below:
<Complex name="Root" type="SaveGameInfo, Assembly-CSharp"> <Properties> <Simple name="PlayerName" type="System.String, mscorlib" value="Celina" /> <Simple name="MapName" type="System.String, mscorlib" value="AR_0301_Ondras_Gift_Exterior" /> <Simple name="SceneTitle" type="System.String, mscorlib" value="Ondra's Gift" /> <Simple name="SceneTitleId" type="System.Int32, mscorlib" value="5" /> <Simple name="Chapter" type="System.Int32, mscorlib" value="2" /> <Simple name="RealtimePlayDurationSeconds" type="System.Int32, mscorlib" value="84865" /> <Simple name="PlaytimeSeconds" type="System.Int32, mscorlib" value="2947900" /> <Simple name="TrialOfIron" type="System.Boolean, mscorlib" value="False" /> <Simple name="RealTimestamp" type="System.DateTime, mscorlib" value="12/13/2015 02:18:17" /> <Simple name="SessionID" type="System.Guid, mscorlib" value="81b92ceb-0faf-4ca2-bfbf-396026a95dd9" /> <Simple name="FileName" type="System.String, mscorlib" value="" /> <Simple name="SaveVersion" type="System.Int32, mscorlib" value="2" /> <Simple name="GameComplete" type="System.Boolean, mscorlib" value="False" /> <Simple name="UserSaveName" type="System.String, mscorlib" value="lef2" /> <Simple name="Difficulty" type="GameDifficulty, Assembly-CSharp" value="Normal" /> <Simple name="ActivePackages" type="ProductConfiguration+Package, Assembly-CSharp" value="BaseGame" /> </Properties> </Complex>
With that in mind we can now take a look at the 2 important functions of the PoE game entry definition.
def POESaveName(f): """ Pillars of Eternity saves are actually zip archives. We are interested in the saveinfo.xml file inside the archive since that contains the save game name we use in-game. Ignore autosaves. """ name = basename(f) res = name.rpartition(" ") if res == "" and res == "": # malformed name return "" if res.startswith("autosave_"): # ignore autosaves return "__IGNORE__" # If we get here it's a user save. Get the user save name archive = zipfile.ZipFile(f, 'r') xmldata = archive.read('saveinfo.xml') root = ET.fromstring(xmldata) ret_name = "" for p in root.findall('Simple'): if p.get('name') == 'UserSaveName': ret_name = p.get('value') break return ret_name def POESaveTime(f): """ Pillars of Eternity saves are actually zip archives. We are interested in the actual save was performed by the player. the saveinfo.xml file inside the archive since that contains the time Returns 0 if there is an error or the unix timestamp if it's determined """ archive = zipfile.ZipFile(f, 'r') xmldata = archive.read('saveinfo.xml') root = ET.fromstring(xmldata) ret_ts = 0 for p in root.findall('Simple'): if p.get('name') == 'RealTimestamp': sdate = p.get('value') ret_ts = time.mktime(datetime.datetime.strptime( sdate, "%m/%d/%Y %H:%M:%S").timetuple() ) break return ret_ts
Those 2 functions get the zip contents, read the xml and return the in-game name of the save and the actual timestamp at which the user made the save. Specifically
<Simple name="UserSaveName" type="System.String, mscorlib" value="lef2" /> and
<Simple name="RealTimestamp" type="System.DateTime, mscorlib" value="12/13/2015 02:18:17" /> from the example save file we posted above.
Armed with the save name and save time the Tool can make a very safe and informed decision on which save file to sync, which ones to keep and which ones to overwrite.
Running the tool periodically
If you need some extra automation you can make the script run periodically. This very much depends on your Operating System. As I have ArchLinux I will talk about using systemd to accomplish this.
Make sure you have read the corresponding systemd documentation and tutorials. Nice links can be found here and here. Assuming we have a systemd hourly target then having a service that will run the sync each hour is as simple as having this service file:
[Unit] Description=Runs the Tool that synchronizes game saves with Dropbox every hour Wants=timer-hourly.timer [Service] User=lefteris Nice=19 IOSchedulingClass=2 IOSchedulingPriority=7 ExecStart=/home/lefteris/.local/bin/steamsaves_sync.py [Install] WantedBy=timer-hourly.target
Place this in
/etc/systemd/system/saves_sync.service and make sure that the syncing tool is under
timer-hourly.target should also be placed in the
Simply enable by
systemctl enable saves_sync.service and enjoy your saves being synced every hour.
At the moment this script can be used for any game as long as you fill out the GameEntry properly. I would not recommend using it with the default functions though for 2 reasons.
- The name of the save file may be the same for all saves or just have an increasing timestamp identifier in the save file name like Pillars of Eternity does. In this case the tool would always consider each save different.
- Using the last modification date to determine which save file is dangerous. If you copy/overwite any of the save files around manually or if Dropbox does, then their modification date will change and then the tool will consider them as newer even if they represent an older save. As a result your new saves may be overwritten.
For the above I recommend using the script only with Games for which you have written proper callbacks just like I showed above for PoE.
And finally let me close with the warning that you are using this tool AT YOUR OWN RISK and that it may very well overwrite your saves. Always make a backup of the saves before you start using it.
Hope this tool turns out to be useful for some of you guys. As usual if you have any questions, fire away in the comments section.