-
Notifications
You must be signed in to change notification settings - Fork 1.4k
perf: eliminate redundant detect_language API calls #4716
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Remove explicit Google Cloud detect_language() calls — the translate API already auto-detects source language for free. This eliminates ~43% of Translate API spend (every foreign-language segment was billed 2x). Changes: - Remove _detect_with_google_cloud() from translation.py - detect_language() now uses only free langdetect library - translate_text() returns TranslationResult(text, detected_language_code) capturing the free detection from the translate response - translate_text_by_sentence() aggregates detected language across sentences - transcribe.py: replace brittle translated_text == segment_text check with detected_language_code == translation_language - Clean up unused import in translation_cache.py - Add 16 unit tests covering all new behavior Co-Authored-By: Claude Opus 4.6 <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request makes excellent progress on optimizing API costs by removing redundant language detection calls and improving the logic for handling same-language text. The introduction of TranslationResult is a clean way to pass more data from the translation service. However, the review identifies critical issues regarding concurrency and blocking I/O. The translation methods use synchronous network calls, which will block the asyncio event loop, and shared caches are accessed without proper locking, creating race conditions. Addressing these is crucial for application stability and performance.
Address review feedback: the aggregation logic now requires ALL sentences to return a non-null detected_language_code AND agree, preventing a single detected sentence from skipping translation when others failed detection. Co-Authored-By: Claude Opus 4.6 <[email protected]>
|
Addressed review feedback (commit 6f858e9):
Added test by AI for @beastoin |
…, split_into_sentences Address tester feedback — 12 new tests covering: - split_into_sentences boundary cases (empty, single, commas, whitespace) - detection cache hit avoids re-calling langdetect - detection cache eviction at MAX_DETECTION_CACHE_SIZE - TranscriptSegmentLanguageCache: sticky false, empty text, delete, undetectable 29 total tests, all passing. Co-Authored-By: Claude Opus 4.6 <[email protected]>
…tests Address final tester feedback — 3 additional tests: - Repeated is_in_target_language calls hit sticky cache (no re-detect) - Cache key uses cleaned text when remove_non_lexical=True - Undetectable (None) results are not cached 32 total tests, all passing. Co-Authored-By: Claude Opus 4.6 <[email protected]>
|
Live local dev test results (real Google Cloud Translate API):
API call summary:
by AI for @beastoin |
Summary
Closes #4712 (sub-issue 1/4 of #4651)
detect_language()Google Cloud API calls — the translate API already auto-detects source language for free in every call, but we were paying $20/M chars for redundant explicit detectionTranslationResultfromtranslate_text()withdetected_language_codecaptured from the translate response (zero extra cost)translated_text == segment_textwithdetected_language_code == translation_language(Google normalizes punctuation/spacing, causing false negatives)Impact
~43% reduction in Google Translate API spend — every foreign-language segment was being billed 2x (once for detect, once for translate).
Changes
backend/utils/translation.py_detect_with_google_cloud(), addTranslationResultNamedTuple,translate_text()returnsTranslationResultwith detected language,translate_text_by_sentence()aggregates detection across sentences (all must agree),detect_language()uses only free langdetectbackend/utils/translation_cache.pysplit_into_sentencesimportbackend/routers/transcribe.pydetected_language_code == translation_languageinstead of text equalitybackend/tests/unit/test_translation.pyTest plan
detect_languageAPI calls in Cloud Console after deployRisks / edge cases
langdetectis less accurate than Google Cloud for very short text (≤5 words) — but this is a pre-filter only; translation still happens and catches it viadetected_language_codedetected_language_codeisNonewhen sentences disagree, so translation is preserved (safe default)by AI for @beastoin