Meilisearch 1.12.7 OOMs and Crashloops when Bar-Assistant Connects

[reefland]

FYI - something changed with Meilisearch v1.12.7 from version 11. It used to start just fine with a RAM limit of 384Mi. I did the normal upgrade steps (stop, wipe volume, start, etc) and it got OOM killed. No problem, bumped it to 512Mi and it started. Logs look clean:

Config file path: "none"
Database path: "./data.ms"
Server listening on: "http://0.0.0.0:7700"
Environment: "production"
Commit SHA: "e20b91210d1b5da82072289a9198435276661174"
Commit date: "2025-01-23T13:41:34Z"
Package version: "1.12.7"
Anonymous telemetry: "Disabled"
A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key.
Check out Meilisearch Cloud! https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli
Documentation: https://www.meilisearch.com/docs
Source code: https://github.com/meilisearch/meilisearch
Discord: https://discord.meilisearch.com
2025-01-24T00:51:04.169377Z INFO actix_server::builder: starting 8 workers
2025-01-24T00:51:04.169401Z INFO actix_server::server: Actix runtime found; starting in Actix runtime
2025-01-24T00:51:13.491270Z INFO HTTP request{method=GET host="10.42.6.32:7700" route=/health query_parameters= user_agent=kube-probe/1.31 status_code=200}: meilisearch: close time.busy=258µs time.idle=13.3µs
2025-01-24T00:51:13.985094Z INFO HTTP request{method=GET host="10.42.6.32:7700" route=/health query_parameters= user_agent=kube-probe/1.31 status_code=200}: meilisearch: close time.busy=39.6µs time.idle=11.3µs

Then tried to start bar-assistant and Meilisearch is OOM killed instantly. I bumped Meilisearch to 1Gi RAM and again OOM killed as soon as bar-assistant tried to connect. I had to remove the resource limits on Meilisearch in order for bar-assistant to connect and complete its sync.

It seems it's just the initial in-rush of connections/data during the initial sync causes it. A heads up if others now have a problem. Perhaps a rate limit is needed?

[zhdenny]

I'm on Meilisearch docker tag v1.12.0-rc.5

No issues here. Just FYI

[reefland]

I agree, it works when no limits are applied. What RAM limits have you set when stating "no issue" ???

[zhdenny]

I agree, it works when no limits are applied. What RAM limits have you set when stating "no issue" ???

That's my point.....I haven't set ANY ram limits..... whatever the default is just works. You can refer here to my entire docker compose stack for SaltRim/Bar Assistant. I never have had to mess with RAM limits while using this app for over a couple years now.

https://github.com/zhdenny/bar_assistant_database/blob/main/docker_compose

[karlomikus]

I'll test the new meilisearch. I think laravel scout can chunk large collection with some config.

The cloud version is running on v1.11 and I haven't noticed any memory issues, but for prod I also use separate queue worker to handle search syncing.

Do you know how many total cocktails and ingredients are you trying to sync with 512Mi of configured ram?

[reefland]

Do you know how many total cocktails and ingredients are you trying to sync with 512Mi of configured ram?

[tommyalatalo]

I just had this same issue yesterday, on a completely fresh install of bar-assistant the meilisearch container was peaking at around 3Gb of RAM usage and then settled in idle at about 2Gb.

[damitjimii]

I just updated to 1.16.0 and everything seems fine; I did have to delete everything in my search data folder but it rebuilt with zero issues, and with no cpu, ram or io spikes.
I should say that I'm running this on a 32GB VM (proxmox 8), and the storage is all older/slow-ish NVME and PCI 3.

As near as I can tell, it's safe to delete the meilisearch data and restart your your containers (stop them first of course). Let me know if I'm wrong.

Below is my current memory use;

root@docker04:/var/lib/docker_configs# docker stats 656a2808e6db --no-stream --format "{{.Name}}: {{.MemUsage}}"
salt-rim: 3.246MiB / 8.459GiB
root@docker04:/var/lib/docker_configs# docker stats 656a2808e6db --no-stream --format "{{.Name}}: {{.MemUsage}}"
salt-rim: 3.246MiB / 8.459GiB
root@docker04:/var/lib/docker_configs# docker stats 00d052c5a0d8 --no-stream --format "{{.Name}}: {{.MemUsage}}"
meilisearch: 81.14MiB / 8.459GiB

Seems kinda low?

[reefland]

@damitjimii - yeap, stop containers, delete data, restart containers to upgrade Meilisearch.

Keep in mind this issue only happened when RAM limits are applied to the container. If you don't have them set then this issue does not apply. Yes, when you check the container it uses low amount of RAM. Within the initial inrush of data when bar-assistant populates Meilisearch which lasts briefly (seconds? less?) is where RAM usage skyrockets hits the limit and container is OOM killed. If the limit is not set, then Meilisearch is free to use as much as it needs for that brief moment and all is well.

[damitjimii]

So while I do not limit my memory; I also didn’t see any spikes. I suppose it could be fast enough it didn’t get sampled but unlikely. I think there is a bar:meilisearch-rebuild -f option in the php app; you could test with that. Sent from my way-cooler-than-your-device device!

…

On Tue, Aug 12, 2025 at 4:51 PM Richard Durso ***@***.***> wrote: *reefland* left a comment (karlomikus/bar-assistant#407) <#407 (comment)> @damitjimii <https://github.com/damitjimii> - yeap, stop containers, delete data, restart containers to upgrade Meilisearch. Keep in mind this issue only happened when RAM limits are applied to the container. If you don't have them set then this issue does not apply. Yes, when you check the container it uses low amount of RAM. Within the initial inrush of data when bar-assistant populates Meilisearch which lasts briefly (seconds? less?) is where RAM usage skyrockets hits the limit and container is OOM killed. If the limit is not set, then Meilisearch is free to use as much as it needs for that brief moment and all is well. — Reply to this email directly, view it on GitHub <#407 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APQ6BT6WTZUBUP6SIWISHIT3NJOXDAVCNFSM6AAAAABVZXNWMOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCOBRGE2DIMZRGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>