-
Notifications
You must be signed in to change notification settings - Fork 850
feat: cgroup-aware machine stats #7488
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: cgroup-aware machine stats #7488
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
9d47058 to
bbd42ed
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds automatic detection of containerized environments (Docker, Kubernetes, Podman) to display accurate resource statistics when running marimo in containers with resource limits. Instead of showing misleading host-level metrics, the footer now displays container-specific memory limits when cgroup constraints are detected, with an updated label to indicate "container memory" vs "computer memory".
Key Changes:
- Added cgroup v1 and v2 detection functions to identify and read container resource limits from Linux cgroups
- Modified the
/api/usageendpoint to return container statistics when limits are detected, adding anis_containerboolean flag - Updated the frontend machine stats component to display appropriate memory labels based on container context
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
marimo/_utils/health.py |
Added has_cgroup_limits() and get_container_resources() functions to detect and read cgroup v1/v2 resource limits |
marimo/_server/api/endpoints/health.py |
Modified /api/usage endpoint to use container resources when available and include is_container flag |
tests/_server/api/endpoints/test_health.py |
Added assertions to verify is_container field is present and boolean in API response |
packages/openapi/src/api.ts |
Added optional is_container boolean field to memory response type |
packages/openapi/api.yaml |
Added is_container boolean field to memory schema definition |
frontend/src/components/editor/chrome/wrapper/footer-items/machine-stats.tsx |
Updated memory stats tooltip to show "container memory" vs "computer memory" based on is_container flag |
Comments suppressed due to low confidence (11)
marimo/_utils/health.py:204
- File is opened but is not closed.
open("/sys/fs/cgroup/memory.max", encoding="utf-8")
marimo/_utils/health.py:213
- File is opened but is not closed.
open("/sys/fs/cgroup/cpu.max", encoding="utf-8").read().strip()
marimo/_utils/health.py:226
- File is opened but is not closed.
open(
"/sys/fs/cgroup/memory/memory.limit_in_bytes",
encoding="utf-8",
)
marimo/_utils/health.py:238
- File is opened but is not closed.
open("/sys/fs/cgroup/cpu/cpu.cfs_quota_us", encoding="utf-8")
marimo/_utils/health.py:345
- File is opened but is not closed.
open("/sys/fs/cgroup/cpu.max", encoding="utf-8")
marimo/_utils/health.py:366
- File is opened but is not closed.
open(
"/sys/fs/cgroup/cpu/cpu.cfs_quota_us", encoding="utf-8"
)
marimo/_utils/health.py:374
- File is opened but is not closed.
open(
"/sys/fs/cgroup/cpu/cpu.cfs_period_us",
encoding="utf-8",
)
marimo/_utils/health.py:323
- File is opened but is not closed.
open(
"/sys/fs/cgroup/memory/memory.usage_in_bytes",
encoding="utf-8",
)
marimo/_utils/health.py:287
- File is opened but is not closed.
open("/sys/fs/cgroup/memory.max", encoding="utf-8")
marimo/_utils/health.py:315
- File is opened but is not closed.
open(
"/sys/fs/cgroup/memory/memory.limit_in_bytes",
encoding="utf-8",
)
marimo/_utils/health.py:292
- File is opened but is not closed.
open("/sys/fs/cgroup/memory.current", encoding="utf-8")
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's a great improvement :). The frontend looks good to me.
Curious if you have tried this solution on the backend? giampaolo/psutil#1011. api docs: https://psutil.readthedocs.io/en/latest/index.html#psutil.PROCFS_PATH
frontend/src/components/editor/chrome/wrapper/footer-items/machine-stats.tsx
Outdated
Show resolved
Hide resolved
|
this is a good find @Light2Dark . @Ddfulton using https://psutil.readthedocs.io/en/latest/index.html#psutil.PROCFS_PATH should hopefully clean up some code, if you are open to make the change |
9d1d710 to
7636ba8
Compare
|
@Light2Dark @mscolnick Thanks for taking a look! I gave I think that thread was actually about the reverse — folks wanting to get host stats from inside a container by mounting the host's When I looked at how Docker/Kubernetes tooling handles this and found they read cgroup files directly (example from cAdvisor). The cgroup paths are part of the stable Linux kernel ABI (v1 since 2008, v2 since 2016), but I agree the file-reading approach is more verbose than ideal. To make it more robust, I've added:
|
I think the suggestion from the thread is to change import psutil
psutil.PROCFS_PATH = “/path/to/container/procfs”However, looking at some of the threads, it seems there is no standard to do ^, and the host & container are not independent so it would not give the container information. Cmiiw. It's a rabbit hole of an issue lol. If so, I think your solution makes sense.. Separately, I do see some users reporting that they want the reverse...which means it's working correctly? Not sure how it's working for them and not here. |
|
With you 100%. I think that's exactly right — a container-specific Not familiar with that Schema Breaking Changes Check above and not clear that it's from this PR — let me know if there's anything I can do to resolve it! |
8849ecd to
66a9501
Compare
…ed units of microseconds when applicable
for more information, see https://pre-commit.ci
7e43f9c to
08d049a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tq! 🚀. you have some failing imports (see CI)
897ec5c to
b2a4139
Compare
c29d356 to
00d5a2f
Compare
|
Fixed the test imports. Bear with me, looking into the sql tutorial failure 🤦 |
e6a7b90 to
6dc1701
Compare
|
Re CI: Tutorial check and and |
|
@Ddfulton yes there are unrelated failing tests |
dmadisetti
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test looks unrelated (I'll look into it)
Thanks for your perseverance and happy New Year!
|
Echoing Dylan—thanks for this! And sorry for all the reviews; the PR looks great now. |
📝 Summary
When running marimo in containers with resource limits (e.g., docker run --memory=16g), the machine stats footer shows host resources instead of container resources. This PR auto-detects containerized environments with cgroup limits and displays container resources instead.
Before: Shows host memory (512GB) even in a 16GB container

After: Shows actual container limits with "container memory" label

This is something we've been doing at Carolina Cloud for a while and have found very useful. Not sure whether it belongs in the main source code but figured I'd share to see what the community thinks.
Also did this for CPU % by using a method similar to
psutil(computing average usage over a time period).🔍 Description of Changes
Backend:
Frontend:
Compatibility:
📋 Checklist