Cloud-jumping: swap hosted services easily with Manifold
Using some part of the cloudâwhether AWS, Azure, Google Cloud, or othersâhas become a standard part of modern web development. RightScaleâs 2018 State of the Cloud reported 97% of respondents using at least one cloud service.
In most instances, itâs easiest to buy into one cloud ecosystem so you have fewer things to manage. But when prices of a service increase, the platform changes, or you need to add an outside service to the mix, how do you handle switching **one **service from your vendor-locked cloud, let alone several?
Manifoldâs free developer tools provide everything you need to build-your-own cloud from existing providers. To see how, Iâll be taking an example application with authentication, hosting, and account email services, and moving the app off AWS and onto independent providers for all three parts.
The App
Our example app is nothing more than a login with an authenticated page, and a password reset via email. So weâre using the following services:
- AWS Elastic Beanstalk (Node.js)
- AWS RDS Postgres
- AWS Simple Email
You can see the code for the starting application
here, in the [master](https://github.com/manifoldco/manifold-sample-app)
branch.
The credentials for the services are all fed in via environment variables through a .env
file.
Though the values arenât saved in the app, you can see all the variable names needed in the
[.env.example](https://github.com/manifoldco/manifold-sample-app/blob/master/.env.example)
file.
Moving to Manifold
With Manifold, you can hook up any service you wantâeven those not listed on the Services page! But for this example, our usecase is satisfied by the existing services of Mailgun and JawsDB Postgres, so weâll use those two. But again, once the application uses Manifold, those can be swapped out easily at any time.
To start, sign up for a free manifold account (only takes a few seconds if you authenticate with GitHub). Then, from our project window, weâll run the following:
Mac (via Homebrew):
brew install manifoldco/brew/manifold-cli
Linux/Windows:
curl -o- https://raw.githubusercontent.com/manifoldco/manifold-cli/master/install.sh | sh
Note: you can read more about this install script on the Manifold CLI Guide.
Then, log in with manifold oauth -- github
if you registered via GitHub, or manifold login
if
you signed up with email / password. Now comes the fun part!
Swapping RDS with JawsDB
Run the following to add a resource, and the friendly CLI will walk you through provisioning (it even gives you pricing and descriptions in-terminal!):
manifold create
For JawsDB, selectjawsdb
as the service type, and kitefin
for the lowest-cost plan (you can
adjust the plan later from the Dashboard).
Complete the remaining prompts like region
and name
with whatever you wish.
Updating the creds
Our app is still using the RDS creds in .env
, so weâll need to update those. Those can easily be
grabbed from within the Dashboard UI, but weâll stick to the CLI
for simplicity. Simply run:
manifold export
âŚand youâll see the credentials exposed into your terminal session (if you need to output a project,
run manifold export --project projectname
). We already have an .env
already, but for future
reference, you can optionally specify an output file like so:
manifold export > .env
What you should see from JawsDB is all the credentials rolled into one URL:
JAWSDB_URL=postgres://$USER:$PASSWORD@$HOST:$PORT/$DATABASE
Pluck apart those values from the URL into .env
to swap your app.
Migrating the data
Thereâs no GUI for exporting your AWS Postgres database, but the standard pg_dump will work just fine. Assuming your RDS instance is publicly accessible, run:
pg_dump $AWS_DATABASE -h $AWS_HOST -U $AWS_USER -f development.dump
Youâll be prompted for your password so itâs not saved in your shell history. To output a
development.dump
file from AWS. Then simply import into JawsDB:
psql $JAWS_DATABASE < development.dump -h $JAWS_HOST -U $JAWS_USER
Youâll likely see some errors like ERROR: role does not exist
because of username differences, but
it should succeed nonetheless. You can verify by running
psql $JAWS_DATABASE -h $JAWS_HOST -U $JAWS_USER
and \dt
to show a list of tables. You can follow
up with a query to double-check the data made it (e.g.: SELECT * from users;
).
Swapping Simple Email (SES) with Mailgun
Migrating our database was about as easy as it gets, but not all service-swapping is quite as rosy.
To switch from Mailgun, we will have to modify app code a bit in addition to a simple .env
update.
Add Mailgun to your Manifold account, either through
running manifold create
locally, or via the Dashboard. Their free
tier is perfect for testing.
Once youâve added it, click the Open Mailgun Dashboard button at the top of the screen:
From there, youâll want to Add Custom Domain:
Follow the instructions that follow for adding Mailgun to your domain, which is necessary to avoid
spam filters and whatnot. Back in our app, weâll swap node-ses
with mailgun-js
:
yarn remove node-ses<br>yarn add mailgun-js
Then in our email config, weâll make a few changes that swap out the SES client with Mailgunâs (you
can see the diff on the
Pull Request from the sample app).
Overall, not too bad! The API, fortunately for us, ended up being similar. We only renamed two env
variablesâso as to not be confusing laterâ and had to change message
to text
in our email call.
Once some test emails are sent, weâre pretty much in business!
Swapping Hosting
Our app was originally hosted on Amazon Elastic Beanstalk, and moved to Zeit Now. I expected to glean some insight from migrating hosts, but came up shortâthe process was unremarkably quick and painless. The only lesson I learned is so glaringly obvious, itâs almost not even worth saying:
If you use cloud services that work on any host, you can deploy to any host at any time.
_âDuh,â _youâd say. And youâre not wrong! But itâs not bad advice to keep in mind, either.
đ Tip: Projects and Teams save a ton of headache
Projects are the best way to associate related resources together, especially if youâre using several in the same application or suite of apps. You can create new projects with:
manifold projects create
Associate resources to that new project by running:
manifold projects add
The biggest advantage to projects, besides organization, is one-click **.env**
file
downloads. Clicking either the âdownload .envâ or âShow credentialsâ buttons will yield all values
for all resources in a project. Huge time-saver when it comes to updating access tokens!
đ Even more security
If you want to make it even more secure and load secrets from memory,
try using [manifold run](https://docs.manifold.co/docs/cli-quickstart-6JMEw1CD6wguwIYymUuAQ6#run-your-project)
to start your appâyou can inject env vars directly from Manifold without any of them living in your
file system! The sample app has an
example of how to do this in [package.json](https://github.com/manifoldco/manifold-sample-app/blob/manifold/package.json#L9)
.
đŻâ Team projects
Team projects are even more useful. With team projects, you can invite and uninvite other developers
to access shared resources, and pay for it all with one set of billing info. To create a team
project (assuming youâve created a team with manifold team create
), first switch to a new team:
manifold switch
And then you can run manifold projects create
from that new team context to create a project for
that team (currently, you canât convert an individual project to a team project, but itâs not too
bad to move existing resources from an old project to a new oneâyou donât have to start from scratch
again).
Recap
Still debating how much effort itâd be to migrate cloud services for your app? Here are some good and bad practices that should give some indication:
Quick & easy migrations
- Environment variables used for all credentials
- Open-source technology powers most of the app
- Configurations are all abstracted into a central place
Painful migrations
- Hard-coded credentials everywhere
- âBlackboxedâ or proprietary systems power most of the app
- Configurations are written ad-hoc, wherever theyâre needed
Migrations may not ever be fun, but with tools like Manifoldâs CLI it can come pretty darn close.
The best migration may be none at all, but when the time inevitably comes, Manifold makes it as painless as it possibly can be.