Best Practices to Migrate Changes to a Live FileMaker Server
FileMaker Migration Assistant Tool vs. Manual:
Best Practices to Migrate Changes to a Live FileMaker Server
Tips from 360Works Senior Developer and Product Manager, Joe Martin
We are writing this article to share approaches to upgrading, modifying, or making changes to a production FileMaker file on FileMaker Server. In the past, FileMaker developers haven’t had many options for rolling out new changes to their live databases. Even if you were cautious, and did your development in a separate Dev copy of your production file, it was difficult to migrate these changes to your production file.
You could roll your own deployment workflow, scripting FileMaker imports for each table in your solution, and update this workflow whenever you added new tables or fields. These imports are slow and tedious though, and most of us would simply make the changes directly in the live file. I’m certainly guilty of this. For really minor changes, this is arguably still a good option.
It’s overkill to re-deploy your entire solution to simply fix a spelling error on a field label. However, implementing workflow-altering changes in a live database can be very risky. If a user tries to use an invoice processing script while you are in the middle of development and testing, records could be left in an unknown state, and invoices could be entirely missed by you or your client.
How to Deploy Changes Manually (without an Assistant or Automation Tool)
With the release of FileMaker 17, FileMaker came out with the FMDataMigration tool, completely replacing the need for using FileMaker native imports to move your data around. Here at 360Works, we built 360Deploy around the FMDataMigration tool to streamline and automate its use, making deploying changes a tedium free experience.
Let’s take a look at how you would use the FMDataMigration tool:
To use the FMDataMigration tool, we need to get a copy of it. You can purchase it as part of the FileMaker Developer Subscription for $99 on FileMaker’s website.
Once you have that downloaded, unzip the file and you should see the executable file:
The FMDataMigration tool is a command line tool, so we need to open up terminal to make use of it. To get a sense of how it works, run the FMDataMigration tool with no arguments to see the help text it returns:
To make these terms a little less ambiguous, the “source file” is your production file, the file with all of the data, but none of the new changes you want to add to it. The “clone file” is your development file, the file with the changes you’ve implemented, but none of the data. Note that the FMDataMigration tool requires the “clone file” to be a clone of your Dev file. This needs to be empty so that it can be populated with the data from your Prod file.
To run a very simple migration, I’m going to use my FileMaker files: MySolution_DEV, and MySolution_PROD. I have made a clone of my Dev file (MySolution_DEV clone.fmp12) that I will use in this example. Now we just need to string together the command, which will look something like this:
./FMDataMigration -src_path MySolution_PROD.fmp12 -src_account admin -src_pwd admin -clone_path MySolution_DEV clone.fmp12 -clone_account admin -clone_pwd admin
Run that in terminal to execute the migration:
Success! We now have a new file, MySolution_PROD migrated.fmp12, created next to our old production file. This newly created file contains the data from MySolution_PROD, and the scripts, schema, and layouts from MySolution_DEV. This tool is also blazingly fast, running a migration in less than 1 second. Of course, that is not a realistic example because these files are as basic as they come, but larger files in the neighborhood of 2GB-15GB will finish within 10-15 minutes.
So, thats the basic premise, now let’s apply this to a real world scenario, and walk through a manual deployment using the FMDataMigration tool.
Say you have you development files on a development server, and your production files on a production server. Users are logging in and working on the production server, creating new records, and we want to interfere as little as possible.
- Get clones of the dev files. Remember that clones are needed for the FMDataMigration tool? Let’s make those first. You can generate these from FileMaker or from FileMaker Server
- From FileMaker: File -> Save a Copy As… -> Be sure to specify Type: clone (no records) -> Save
- From FileMaker Server: Create a new backup schedule. In the “Additional Settings” section, check the “Clone” box. This will automatically generate clones as well as backups for your files. Unfortunately there isn’t a way to only generate the clones, you have to create the full backups as well.
- Upload clones to the production server. Now that we have the clones, we need to upload these to the production server. Use any upload tool you like: Google Drive, DropBox, SuperContainer, etc. Once uploaded, download them to the production server.
- Organize your files and prepare for migration. I find it very helpful to create a folder structure that mirrors my needs. We start off with two copies of the same file, and we will be generating a third, and we don’t want to mix any of these up. Also, get the FMDataMigration tool handy, as we will be using it in the next step. Move the files into a folder structure like below. Note: the migrated folder is empty now, but that will be our target for where the migrated file is created:
- Run the migration. We need to string together our FMDataMigration command. We will also be using the “-target_path” parameter to tell the FMDataMigration tool that we want our migrated file to wind up in our migrated directory. This also gives us the opportunity to name it the same thing as the old prod file. Putting together the command will look like this:
./FMDataMigration -src_path prod/MySolution_PROD.fmp12 -src_account admin -src_pwd admin -clone_path dev/MySolution_DEV clone.fmp12 -clone_account admin -clone_pwd admin -target_path migrated/MySolution_PROD.fmp12
Run this command to execute the migration:
Now look at our folder structure, and there will be a new file in the “migrated” directory. This is the newly created migrated file:
- Host the migrated file. Move the newly created migrated file back into the location where you originally pulled the production file from. You will want to make sure it is put back into the same place so that external containers are properly resolved. After it is in place, re-open the file in the FileMaker Server Admin Console.
In this example we migrated a single file. If you need to migrate multiple files, it will compound these steps a bit, as you will need to upload clones, and generate a new FMDataMigration command for each file. Another simplification we made above was assuming that you are confident remoting in to your development and production servers to generate the clones and run the FMDataMigration commands.
The main problem with this approach is the fact that it requires a lot of management, and leaves a lot of room for human error. A bad scenario would be that some step is missed, or a file is left out, and the whole process needs to be restarted. A worse scenario is that a minor mistake causes you to fall outside of your scheduled downtime window, and you need to put the old production file back up, wasting the time that you spent entirely.
360Deploy was built to solve exactly this problem. 360Deploy automates all of these steps so you don’t need to worry about missing a step or generating the proper command. Let’s take a look at a deployment using 360Deploy:
How to Handle Deployment with 360Deploy (Automating an Assistant Tool for FileMaker Data Migration)
360Deploy can be downloaded from our online store here.
To deploy a multi-file solution to a single server, 360Deploy costs $195. This purchase includes the FMDataMigration tool on the back end. When you take into consideration the time spent doing a manual deployment, 360Deploy becomes the most cost effect solution after 2-3 deployments.
Before we can use 360Deploy, we need to install some software on the servers, and setup the 360Deploy configuration. These steps will most likely only need to be done once, whereas with a manual deployment you will be repeating each step with every deployment.
- Install the 360Deploy Server Component on your development and production servers. 360Deploy communicates with FileMaker server using the fmsadmin command line, so it needs to be installed on the same machine as FileMaker Server. Remote into your server, and run the installer included with the download to install the 360Deploy server component. If you aren’t sure if your server needs to have the server component installed, ask yourself: Will 360Deploy need to communicate with this server? If the answer is yes, run the 360Deploy installer on that machine. For example, if this is the server where your dev files live, 360Deploy will need to communicate with that server to create the clones. If this is your prod server, 360Deploy will need to close the files, run the migration, and re-open the files after migration.
- Configure Settings. Open the 360Deploy.fmp12 file included with the download. Click the “Settings” button, and input your license key and registered to name. Or you can skip this and run 360Deploy in demo mode, but you will only be able to deploy files that are named: 360Deploy demo solution.fmp12. Optionally, you can put in an email address, and a report will be emailed to you at the end of the deployment.
- Configure Dev Server. Click the “Dev Server” button, then click “Configure Dev Server”. Follow the steps in the workflow to input your dev server credentials, select your dev files, and store your passwords for the dev files
- Configure Prod Servers. Click the “Prod Servers” button, then click “Add Production Server”. Follow the steps in the workflow to input your prod server credentials, select your prod files, and store your passwords for the prod files
- Run Deployment. Click the “Deploy” button, then click “Deploy” in the new screen.
- Monitor deployment progress. 360Deploy will update the screen as the deployment is running. Once all files have been migrated, you will see the light turn green on for the Server Status, and you will be sent an email if you have provided one earlier.
To repeat this deployment, you only need to click the “Deploy” button again, as your configuration has been saved. 360Deploy handles all the steps for you: creating dev clones, uploading these to the prod server, closing the files, kicking out clients, running the migration, re-opening the newly migrated file.
360Deploy also can be run as a server side script, which can be scheduled in the FileMaker Server Admin Console for a particular date and time if you need to run deployments during off-peak hours.
I can’t fault anyone for going the manual route, and wanting complete control over the deployment process and assurance that they are confident in the process. To that, I want to point out that 360Deploy is rock-solid, with over 5000 deployments since its release 9 months ago.
We are always improving 360Deploy, and would be happy to hear from you if you have any suggestions. Send us a message at firstname.lastname@example.org.
Stay updated with 360Works
360Works builds FileMaker Pro plug-ins and add-ons, such as SuperContainer, MirrorSync, and ScriptMaster, 360Works offers consulting and programming services for FileMaker Pro for businesses worldwide.