For low-traffic applications, Cloud Run is dramatically cheaper than App Engine.
Abstract
I was hosting a small web app as a side-project and looking to spend less money. I started out using Heroku, then moved to Google’s Cloud Platform. Using rigorous methods and markdown tables, I performed a science-inspired “how much does this cost?” comparison between App Engine and Cloud Run. This study finds that Cloud Run is usually the best option, although if you have money to burn are a “price insensitive consumer,” then App Engine is a bit zippier.
Introduction
Imagine you have a side-project-type web app and you’re looking to host it on Google’s Cloud Platform (GCP) but you don’t want to spend too much ca$h. Which GCP service do 4 out of 5 scientists recommend? Let’s find out.
Background
My incredible journey went basically thus: I built a small express app for upcoming Canadian holidays and wanted cheap but usable hosting. Initially, I was using Heroku’s $7/month Hobby Plan because at the end of the day month, it’s only $7. (ie, that’s like 3 coffees: ‘a coffee’ being the base unit of diminutive purchases.)
Heroku was really easy to get going with, to integrate with GitHub Actions, and to ssh into when I needed to fiddle with something. But around month five, it dawned on me that it was going to cost $7/month for the rest of my life, so I started looking for other options.
Pivoting to Google Cloud Platform (GCP)
GCP was the cloud vendor with the most bonus cash on sign-up, so I figured that was a pretty neutral and unbiased reason to pick it. However, as a hapless first-time user, there are a lot of “solutions” to choose from.
It seems like you’re not a real cloud vendor unless you can bury newcomers under an avalanche of vaguely differentiated products with abstract geometrical logos, so a straightforward question like “where do I host a basic express app?” didn’t have an obvious answer.
Cutting through the media bias with facts and logic, I was able to narrow it down by following the research methodology of googling “google cloud how do I host express app”.
The two options that popped up were:
Both services will run apps and I had an app to run. Seemed perfect: they anticipated me like how I anticipated Canadians are looking for information about holidays.
Methodology
By signing up, I was granted 300 (!!) GCP bucks, and as a long-time government employee I knew this meant I had to find a creative way to spend it before the end of the fiscal year. Are you thinking what I’m thinking? Let’s run a research study!
(This is where the science comes in.)
My research question was “Should I use App Engine or Cloud Run to host my fun but unprofitable app?”, and to investigate that I opted for the immersion method where I would assume the role of a developer trying to host an app on Google Cloud.
Setup
As a precursor, I needed to set up my app on both services simultaneously. For the initial setup, I used the Quickstart material provided by Google at no cost to embedded researchers like me. (Both Quicks-start are pretty easy to follow once you have the gcloud command-line tools installed.)
Overview: App Engine (AE)
On AE, my express app runs as a node process, like booting it up with npm start locally. AE is a traditional hosting platform: it runs continuously and serves requests as they come in. At the end of the month, you pay for the amount of time it was running, which is typically “the entire month”.
Overview: Cloud Run
Cloud Run runs containers, so for each release you have to build a container and push it to GCP. Unlike App Engine, Cloud Run only runs when requests come in, so you don’t pay for time spent idling.
Containerized apps are more portable but not always something you focus on during development. It’s worth noting that the Cloud Run Quickstart provides 9 example Dockerfiles depending on your language of choice. (I used the Node.js one as a basis.)
Simulating traffic
At this point in the study, I had 2 instances of my app running:
- In App Engine:
https://hols-ae.nn.r.appspot.com/ - In Cloud Run:
https://hols-hzlcxvebra-ue.a.run.app/
Because real applications have real traffic, I set up a ping service to send requests to each site exactly once every 47 minutes for the rest of time, just like how a Real Human Being™️ would browse.
Having completed my setup, it was time to let the experiment run its course, so I passed the time doing highly academic things like rinsing noobs at dominion.games.
Duration
2 months.
Findings
There were 2 principal findings of the study.
- For a low-traffic application, Cloud Run is dramatically cheaper than App Engine
- App Engine seems to respond slightly faster
1. Ongoing costs — Cloud Run wins ✅
| Cloud Run | App Engine | Heroku Hobby Plan | |
|---|---|---|---|
| Monthly cost | $0.09 | $11.29 | $7.00 |
Wow.
App Engine runs 24/7 for the entire month whereas Cloud Run only runs when serving requests, and the difference is startling.
Previously, I had been paying $7 a month for Heroku’s Hobby Plan.
- App Engine would cost me about 50% more
- Cloud Run costs 99% less, oh my goodness
So basically it’s a blowout win for Cloud Run here.
2. Request latency — App Engine (usually) wins ✅
I also used some online speed test tools to measure the response times of my 2 instances. The results weren’t totally consistent, but App Engine generally responded more quickly.
Pingdom Speed test
(Results of 3 runs from São Paulo)
| Cloud Run | App Engine | |
|---|---|---|
| Run 1 | 632 ms | 471 ms |
| Run 2 | 485 ms | 568 ms |
| Run 3 | 562 ms | 470 ms |
| Average | 559 ms | 503 ms |
Here we see App Engine responding on average 56 ms faster than Cloud Run (although in 1 case, Cloud Run was faster). The huge caveat here is that these times vary widely between runs, sometimes tripling or quadrupling depending on Who The F*ck Knows.
WebPageTest
(Results of 3 runs using “3G” download speed.)
| Cloud Run | App Engine | |
|---|---|---|
| Run 1 | 5.217 s | 5.010 s |
| Run 2 | 5.310 s | 4.922 s |
| Run 3 | 5.353 s | 5.089 s |
| Average | 5.293 s | 5.007 s |
Again, keep in mind that these numbers shift around between runs.
Why is App Engine faster?
This isn’t totally clear to me, but I can speculate.
The one measurable difference I noticed is that that the total request size from Cloud Run was larger because it doesn’t gzip files by default.
| Cloud Run | App Engine | |
|---|---|---|
| Page size | 125.8 KB | 119.4 KB |
The Pingdom Speed Test for Cloud Run recommended I Compress components with gzip, and looking through the requests, my combined .js assets are indeed about 6 KB larger.
Downloading bigger files makes your site slower, but I don’t think that’s the whole story.
The big difference between the two services is that Cloud Run doesn’t run your container unless it’s getting requests. When a request comes in, it does 3 things:
- boots up the container
- serves the request
- shuts down the container
It seems likely that the extra time needed to boot up the container adds to the total request time, leading to an average slower response time from Cloud Run.
Of course, you also save a lot of money doing it this way, so the tradeoff here is whether you care more about optimizing your speed or your cost.
Findings
For me, the findings are decisive. If you’re a hobbyist developer and you want to host your fun app for next-to-free, you should definitely use Google Cloud Run.
However, if money is no object, then you can pay exponentially more per month for a marginal speed boost on App Engine.
Further reading
- Read more about why Google Cloud Run is better than other hosting options
- For an excellent intro to Docker, check out this excellent guide by Robert Cooper
- Check out Google’s “Build and Deploy” Quickstart for Cloud Run
- Use Github Actions to deploy automatically to Cloud Run
Top comments (29)
Your app-engine was setup to "autoscale" hence the instance would stay up constantly costing you $. If you changed it to "basic" auto-scaling, GAE would have auto scale down and stop the instance and costs should be similar to cloud run. Could you pls re-test with this setting so its a more fair comparison. Thanks,
cloud.google.com/appengine/docs/st...
The linked documentation could be more clear, but it is not correct to say that "autoscaled" instances are "up constantly costing you $". You linked to the "Instance State" section, and it is saying that "autoscaled" instances will only ever show as being in the "running" state (vs the "stopped" state possible for "manual" or "basic" scaling). This is because "autoscaled" instances are shut down after some time (if no requests come in), not that the instances are running 24/7.
I have to agree with @msl00 here! It's unclear and AFAIK AppEngine can NOT auto-scale to 0!
App engine standard environment can scale down to zero. I'm paying zero for for my hobby project. cloud.google.com/appengine/docs/th...
The problem with autoscaling to 0 is that it causes cold start. Assume your app is idle state(usually after 20 minutes when no new requests coming) so the total number of instance would be 0. So it take about 10 seconds to start the server in AppEngine for a first request.
I laughed out a loud a couple of times reading this article. You have a great sense of humor and a fantastic writing style :)
"sometimes tripling or quadrupling depending on Who The F*ck Knows."
=))
Nice tests and post, but you should specify which environment you're using for GAE.
I primarily code in Java (Haven't deployed with Node.js on GAE so far) and from my experience the Standard environment works similarly to Cloud Run, as it spins up a new instance when a request is made (if there wasn't one already idle). The instance stays idle for 15 mins after that it's shutted down. Google gives you a daily free usage quota of 28 hours for instances. Hence if you tests were runnning once every 47 mins (and the requests didn't require much processing power)... then your daily cost would be 0.00$ as you wouldn't be surpassing the daily free quota.
If your tests were on the Flexible Environment then that's a whole different story as an instance has to be idle all the time and I am not so sure what machine type they start of from there. In Standard the lowest instance is an F1, whicho would have 256mb RAM, which is not much but enough for a simple app.
Nice post! In the future would be cool to do a benchmark to compare with a similar AWS service.
It's a good question. At work we use Fargate a lot, which I find a lot more complex than Cloud Run to set up, but it has a similar "serverless container" platform concept as CR does.
I was unsuccessful trying to find out something about your app's datasource. Because IMO there is the big cost, when using Cloud Run. I agree with you, Cloud Run is cheap, but you have to use Cloud SQL as a datasource and in my experience it is far expensive compared with a SQL instance running in GCE.
Just to chime in for anyone confused by the huge price difference.
In my experience, App Engine Standard Environment with automatic scaling will effectively scale down to "0 instances": After 15 minutes with no request, with automatic scaling, you are billed NOTHING.
(dev-to-uploads.s3.amazonaws.com/up...)
Google states on their pricing page:
Accrual of instance hours begins when an instance starts and ends as described below, depending on the type of scaling you specify for the instance:
Paul, thanks for this hilarious and informative comparison!
I was wondering if you had also considered deploying this site as a static site on something like Firebase Hosting/Vercel/Netlify/Github Pages, etc. (straight to a CDN, instead of worrying about hosting)? Next.js takes cares of a lot of pain points (data fetching, caching etc.). We're considering something like that for our own site.
You do lose the benefit of having a proper node.js backend, but so far our needs can be met by Next.js mixed with maybe some serverless functions.
Overall, that could maybe bring costs down even further than an auto-scaling container, as long as you don't need access to a real server...?
Just food for thought.
NextJs is great but locks you in to using old hat React, not everybody's cup of tea, mind you.
I’ve been running a couple of applications on App Engine for several years and never been billed more than $0.50 per month. I think you may have set up App Engine incorrectly for the workload you are using.
Misleading post as the premise is that you pay for App Engine 24/7 which isn't true on the standard instances (predefined languages versions e.g. Go 1.12) only if you choose flex (custom versions). Otherwise you have a point but it's not clear and standard App Engine covers most use cases.
Great post! App Engine standard or flex?
What about the 1 million requests are free per month on cloud run?
This is the monthly free tier on Cloud Run:
I am building the same kind of hobby app, and Google lead me here ヽ(^O^)ノ
Thank you very much.
I think I will also use Cloud Run, it's flexible and we can move the whole docker to another cloud later
I've built a startup company using Google Cloud - not sure why you would move to somewhere else that has matching products. Once you have Cloud Run containers, you'll want to use pub/sub for event driven architecture. Combined with realtime Firebase Firestore database you're pretty much in a situation that is not matched anywhere else. Especially for the price.
Yeah, it's a great place to start! Heroku is easier to use but it costs like 100x more money.
Really appreciate the commentary on costs 👏👏
It's a naive article. Highlighting saving 10$. Would appreciate if you could do more research and provide insightful facts about use cases, trade offs and so on.
What service did you use for the database? Cloud SQL?