The first and highest-impact place to optimize is in softwareland: Mongo schema design, indexing, and oplog. Subscription and query design (also try to avoid types of queries that use the polling driver)...

Comments

Serkan DurusoySerkan Durusoy
2 years ago

Deleted

mariomario
2 years ago

No one ever talks about the NodeJs memory limit. "The limit can be raised by setting --max-old-space-size to a maximum of ~1gb (32-bit) and ~1.7gb (64-bit)". - https://github.com/joyent/node/wiki/FAQ

Quoting the article: "I average 8MB per client, so a server with 80GB of RAM would max out by the time I hit 10k connected clients." and "For instance at 50k connected clients, I might have six app servers, each with around 8.3k clients, who use up over 83% of the server’s memory".

Basic math: 50000 clients / 6 server apps = 8333 users x 8 MB per client =~ 66 666,67 MB = 65 GB on a single app server.

Having in consideration your 8 MB estimation and NodeJs limit of 1.7 GB on a 64 bits machine: 50000 clients x 8MB = 400000MB /1024 = 390 GB / 1.7 = 229 servers without counting with Mongodb servers, load balancers, etc.

Conclusion: High-Level Bullshit Guide to Scaling Meteor

Reply

Hi Mario, looks like the max limit was lifted on 64-bit (and 2GB on 32-bit), but I will add a note on that to the guide, thanks!

https://code.google.com/p/v8/issues/detail?id=847

Also let me know if you've measured different numbers for per-connection memory.

Reply

This max limit it very hard to happen with Meteor since our main issue is CPU. So, CPU become an issue before the RAM. Since we can't do native multi-core support, I usually goes with single core instances.

Just to note, this is about the heap size. Actual memory usage could be higher than the heap allocation. So, practically it can go beyond that.

But generally speaking it's better not to use much RAM with node. Since, it's a garbage collecting, with higher amount of RAM we'll have issues.

Reply
Serkan DurusoySerkan Durusoy
2 years ago

A non-bullshit comment would be one not ending with a bullshit conclusion. The meteor community goes to great lengths to keep it open, constructive and helpful.

I doubt that the author has been paid to provide us all with a rock solid, applicable to all, down to earth, vast in detail meteor scaling tutorial. I think he deserves kudos for just being kind enough to write down some piece of information for anyone to "utilize". But as with all technical write ups, it is the readers' responsibility to not blatantly "consume".

I've also yet to see a very extensive and complete guide on scaling meteor, or scaling any stack for that matter. And it is perfectly normal because there are too many layers within the stack and too many variables just changing across apps, usecases and user habits.

I find it great that you've pointed out node's memory limit, which is a good piece of information to consider when planning on scalable meteor deployments. Thank you for surfacing that information.

In the author's defense, he does add that cpu would more likely to be a bottleneck than ram would be, in terms of processing power. He also makes other remarks on database scaling.

Regarding the cpu limits, I've recently deployed Phusion Passenger. While doing that, I've come across an interesting article http://blog.phusion.nl/2013/03/12/tuning-phusion-passengers-concurrency-settings/ that touches concurrency. The interesting part is:

"... if your CPUs are not used constantly, e.g. because they’re often blocked on I/O, then ... increasing the number of processes/threads does increase concurrency and throughput, at least until the CPUs are saturated..."

That means, the common belief around number of cpu's being the limit of number of concurrent node/meteor instances is just precautionary.

One can, in theory, utilize a newish multi cpu multi core server to the extent of its ram capacity.

Again, it all depends on the cpu/memory profile of the application itself on average use case scenarios.

Reply
mariomario
2 years ago

Hi Serkan,

"A non-bullshit comment would be one not ending with a bullshit conclusion. The meteor community goes to great lengths to keep it open, constructive and helpful."

I respect the community and I thankful for it's efforts every day. In my perspective, what I've seen lately is too many packages, articles, etc. talking about how great Meteor, how easy it is to create apps using as few lines of code as possible, etc. For a beginner it's great to have such resources.

When you start to use Meteor daily, you'll find that a lot is missing, both packages, core functionalities and articles. My problem with the article is very simple: It claims to be a guide when, in fact, it doesn't guides you at all. At the end, you won't be able to deploy a meteor application. Even writing "Run meteor deploy free-domain.meteor.com" is more helpful because at the end you actually have an application deployed. It also lacks technical depth and/or real world expertise. I don't expect anyone to do my job but there is a difference between writing something and writing something that adds value. Did you deployed a meteor app with a load balancer? Great! I would love to read an article about it!

I've nothing against the author or the community, my point is: Let's stop writing about how simple it is and how many lines of code we use, in a real world scenario it doesn't really matter the number of lines and there will be always hard problems to solve . What I'm assisting, in my perspective, is pure marketing (Hackers News starts to feel like this also). So many blogs emerging with articles and open source apps, in the end, only a few are actually helpful.

If I've offended someone, specially the author, I apologise. It's not my intention to offend neither the author nor the community. I see 2 ways we can contribute, to add or not to add value, it's just my personal opinion.

Reply
Serkan DurusoySerkan Durusoy
2 years ago

Well, I understand your frustration. I do. In fact, I share your sentiment. And I also appreciate you being vocal about it.

But the way to improvement and getting people to write great posts would be through constructive criticism.

In fact, let's not stop writing how simple it is, and let's not stop writing posts that sell like a guide but is not. But let's improve them when we see them. Let's comment on each post on the post's page about what it is missing, how it can be improved.

Let's foster the discussion, foster the innovation. Mark it right there for future readers that our collaborative vision is much more than writing one-liner apps or "high-level guides".

And look where we are now. Apart from the name-calling, your comment led to us learning about a memory limit that node imposes. And the author followed up on it. Arunoda chipped in with his own pointers. I think that's a great win.

And now I'm sure that when somebody makes it their own to write down a thorough post about scaling meteor, this will be one of the appendices to aid.

Reply

Hi Serkan, thanks for the comment on tone. If I remember correctly from my Rails days (The Dark Ages), traditional ruby server single-threaded processes did block on IO, and handled one request at a time, which means you would want many processes, until you filled up the machine's RAM. Node (even with Fibers and code that looks like it blocks) does not block on IO, and a single process handles as many requests as it can until it takes the whole CPU core. Check out the great section "Limited concurrency at the app server level" in this article: http://blog.phusion.nl/2010/06/09/does-rails-performance-need-an-overhaul/ first talks about Rails, then Node at the bottom.

Reply
Serkan DurusoySerkan Durusoy
2 years ago

Nice read, thanks.

I'm coming from a java background where scalability has not been an issue, but rather a form of art in which optimization is a permanent task.

Being fed up with its bloat, I've dated play and flirted with vertx (which are just great by the way), disliked node for its lack of consistent tooling, and then suddenly fell in love with Meteor due to it ... being Meteor :)

I think there is yet lot to discover in Meteor's scalability and that it has great potential.

But, there is also this great work https://www.techempower.com/benchmarks/ which seems to be now inactive, but surfaces some great data (to take with a grain of salt of course), among which, node being one of the poorly performing frameworks.

Node and Meteor do shine on a lot of grounds, performance being one of the duller aspects. But hey, one cannot expect each and every thing from one stack. Meteor is what it is, and as long as it does the job while I enjoy getting it to do the job, nothing beats it.

And if a project out of every ten requires some specialty tech, then I'll go use that for that one project, no hard feelings.

Reply