Repository navigation
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1153 +/- ##
==========================================
+ Coverage 93.00% 93.16% +0.16%
==========================================
Files 54 54
Lines 3672 3733 +61
==========================================
+ Hits 3415 3478 +63
+ Misses 257 255 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
9554a97 to
1782ca6
Compare
2d70b86 to
c1f6b59
Compare
| run = await run_client.get() | ||
| if run is None or run.status not in _ABORTABLE_STATUSES: | ||
| return | ||
| await run_client.abort(gracefully=True) |
There was a problem hiding this comment.
Is there any benefit in doing GET request for run status instead of sending a blind ABORT anyway? The previous state could probably be guessed from the response to the abort request.
It is important to clean up as quickly as possible to not get aborted midway, so going from 2 requests in series to 1 request is maybe worth it.
There was a problem hiding this comment.
Thanks, done in ba138b3. Each child now gets one abort(gracefully=True) request without a status check first.
Mantisus
left a comment
There was a problem hiding this comment.
From Claude. Could you check that, pls?
Children started after ABORTING are never aborted. abort_runs_with_parent runs once. A named start or resurrect after that isn't covered. A typical orchestrator loop hits this:
for i, batch in enumerate(batches):
await Actor.call(actor_id, batch, run_name=f'batch-{i}', abort_with_parent=True)On a graceful abort, the current child is aborted and call returns ABORTED. The loop then starts the next batch inside the 30 s grace period. Once the parent is force-killed, that child is orphaned. A later call under an existing name can also resurrect the child we just aborted, since the registry resurrects ABORTED runs. I reproduced the first case in a unit test: after emit(ABORTING), a new named start with abort_with_parent=True never gets aborted.
Suggestion: set a "parent aborting" flag in abort_runs_with_parent. After that, find_or_start with abort_with_parent=True shouldn't start or resurrect runs. It could raise, or abort the run right after recording it. A test for this case would help too.
| await self.event_manager.__aenter__() | ||
| # Only the platform emits `ABORTING`, and it does so through `ApifyEventManager`. | ||
| if isinstance(self.event_manager, ApifyEventManager): | ||
| self.event_manager._on_internal(event=Event.ABORTING, listener=self._abort_child_runs) # noqa: SLF001 |
There was a problem hiding this comment.
The method _on_internal was added in that PR. Let's make it public.
There was a problem hiding this comment.
Well, that's the point of them being private. We need to distinguish the SDK's own listeners from the users's.
off() without a listener removes everything a user registered, but the SDK's ABORTING handler that aborts child runs has to survive that.
If {on|off}_internal() were public, users could put their own listeners into that internal group too. Then off() would no longer reliably remove a user's listeners, and the line between SDK listeners and user listeners would be gone.
|
continue here after #1184 |
c1f6b59 to
ed70f26
Compare
Adds
abort_with_parenttoActor.start,Actor.call,Actor.start_taskandActor.call_task. It's off by default and needsrun_name. The flag is stored asabortWithParentin the name's record under__ACTOR_CHILD_RUNS, so a parent that migrated or was resurrected still knows which children to abort. The latest call under a name sets the value.The SDK registers its own
ABORTINGlistener onApifyEventManagerwhen the Actor context starts. It's kept apart from user listeners, sooff()without a listener doesn't remove it. On a graceful abort, it sends a graceful abort to every flagged child, one request each, all in parallel. The platform returns a finished run unchanged, so there's no status lookup first. A child the API no longer knows is skipped, and any other failed abort is logged without stopping the rest. Each child is aborted with the client it was started with in this process, which matters when a customtokenwas passed. A child from before a migration uses the default client.Once the parent is aborting, a named start or call with
abort_with_parent=Trueraises aRuntimeErrorand doesn't start, resurrect or reattach a run. A loop that starts one child per batch stops there, so it doesn't leave a new child behind. A start already in flight whenABORTINGarrives finishes recording its run, and that run is then aborted. Runs started withoutabort_with_parentaren't affected.A hard abort, a timeout or an OOM kill still leaves the children running. Covering that needs a parent/child link on the platform.
Closes: #1128
✍️ Drafted by Claude Code