fix: use bounded read for workflow catalog HTTP responses - #3766
Open
Quratulain-bilal wants to merge 1 commit into
Open
fix: use bounded read for workflow catalog HTTP responses#3766Quratulain-bilal wants to merge 1 commit into
Quratulain-bilal wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Bounds workflow and step catalog HTTP responses to 1 MiB, mitigating memory-exhaustion risks.
Changes:
- Uses
read_response_limitedfor both catalog fetch paths. - Applies
MAX_JSON_METADATA_BYTESas the response limit.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/workflows/catalog.py |
Adds bounded reads for workflow and step catalogs. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
src/specify_cli/workflows/catalog.py:1219
- Please cover the step-catalog fetch path with an oversized streamed response and assert
StepCatalogError. StepCatalog has separate fetch/cache handling, and its current fetch tests also exit before reading the body, so they do not verify that this call site actually enforces the new limit.
data = json.loads(
read_response_limited(resp, max_bytes=MAX_JSON_METADATA_BYTES).decode("utf-8")
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Medium
Comment on lines
+543
to
+544
| data = json.loads( | ||
| read_response_limited(resp, max_bytes=MAX_JSON_METADATA_BYTES).decode("utf-8") |
Contributor
Author
There was a problem hiding this comment.
Added 4 regression tests:
test_oversized_workflow_catalog_response_rejected: monkeypatchesMAX_JSON_METADATA_BYTESto 512, serves 1024 bytes, assertsWorkflowCatalogErrorwith "exceeds maximum size"test_oversized_workflow_catalog_does_not_block_healthy_one: proves a valid catalog still works after an oversized one was rejectedtest_oversized_step_catalog_response_rejected: same pattern forStepCatalog?StepCatalogErrortest_oversized_step_catalog_does_not_block_healthy_one: proves a valid step catalog still works after rejection
mnriem
requested changes
Jul 27, 2026
mnriem
left a comment
mnriem
left a comment
Collaborator
There was a problem hiding this comment.
Please address Copilot feedback
The workflow catalog fetch used unbounded resp.read() to read HTTP responses into memory. A malicious or misconfigured catalog server could return an arbitrarily large response causing OOM. Replace with read_response_limited() capped at MAX_JSON_METADATA_BYTES (1 MiB) at both call sites, consistent with how other JSON fetch paths in the codebase enforce bounded reads.
Quratulain-bilal
force-pushed
the
fix/unbounded-workflow-catalog-response
branch
from
July 27, 2026 22:15
291d343 to
5c2087c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The workflow catalog fetch (
WorkflowCatalog._fetch_single_catalogandStepCatalog._fetch_single_catalog) used unboundedresp.read()to read HTTP responses into memory. A malicious or misconfigured catalog server could return an arbitrarily large response causing OOM.Changes
src/specify_cli/workflows/catalog.py(lines 541, 1214): Replacedresp.read()withread_response_limited(resp, max_bytes=MAX_JSON_METADATA_BYTES)capped at 1 MiB at both call sites, consistent with how other JSON fetch paths in the codebase enforce bounded reads.Testing
All 176 tests in
tests/workflows/pass after the fix.Security Impact
This is a Medium severity fix - it closes a potential memory exhaustion vector against the workflow catalog fetch endpoint.