fix: add AbortController support to OpenAI-compatible providers #10784
+329
−248
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR attempts to address Issue #10779.
Problem
The Stop button in Roo Code does not work for OpenAI Compatible API providers (llama.cpp, LM Studio, etc.). When users press the Stop button, the UI shows stop but the inference continues running on the server because no abort/cancellation signal is sent to the API.
Root Cause
The
OpenAiHandlerandBaseOpenAiCompatibleProviderclasses were missingAbortControllersupport. When the Task class callscancelCurrentRequest(), there was no abort signal being passed to the underlying OpenAI SDK calls.Solution
Added
AbortControllersupport to both handler classes:OpenAiHandler (
src/api/providers/openai.ts):abortControllerpropertycreateMessage()andcompletePrompt()signal: this.abortController.signalto all SDK callsBaseOpenAiCompatibleProvider (
src/api/providers/base-openai-compatible-provider.ts):This mirrors the existing implementation in
OpenAiNativeHandlerwhich already has proper abort support.Testing
Feedback and guidance are welcome.
Important
Add
AbortControllersupport toOpenAiHandlerandBaseOpenAiCompatibleProviderfor request cancellation, updating tests accordingly.AbortControllersupport toOpenAiHandlerandBaseOpenAiCompatibleProviderclasses for request cancellation.createMessage()andcompletePrompt(), create and clean upAbortController.abortController.signalto SDK calls inopenai.tsandbase-openai-compatible-provider.ts.openai.spec.tsandbase-openai-compatible-provider.spec.tsto expectAbortSignalin request options.This description was created by
for 5d606ba. You can customize this summary. It will automatically update as commits are pushed.