|
20297
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20297
|
|
20298
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20298
|
|
20299
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20299
|
|
20300
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20300
|
|
20301
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
selection*SOURCE CONTROV CHANGESy Changes.env.local• .env.nikilocal@ .env.otherE ids.txtraw sal_query.sq* WEBHOOK_FILTERING_IMPLEMENT.# JiminnyDebugCommand.php app/d** SimulateWebhooksCommand.php a..." CreateMockAskJiminnvReportResult..ч* RateLimitException.php app/Excepti..MatchActivitvCrmData.php app/Jobs... M1* Client.php app/Services... U 9 + 9, M"loadina.nno confid* favicon.ico public** Handle-lubsootRateLimitTest.oho t...** CanAccessAiReportsTest.php tests/...** ClientTest.oho tests/Unit/Services/Cr.V GRAPHP9Auto @ ÷J1O JY-20725 add ...• JY-20725 add tests Lukas Kovali!• JY-20725 add HS rate limit handling on a...Merae oulll request #12.Merge branch 'master' into JY-20818...Merae oull reauest #12057 from limin….JY.20818 move ask liminnv renorts to..Merge pull request #12052 from jimin...Morao hranch Imactorl into.IV.20917Merge pull request #12047 from jimin...A IV.20662 roturn woid flier• JY-20662 Removed word_boost para...JY-20817: Fix deleting old tracks mih...Merge pull request #12043 from jimin...Merge branch 'master' into JY-19938...Merge pull request #12044 from jimin...• JY-19938 Added consent information…JY-19938 reorder array map llianJY-19938: Address PR review comme.JY-19938 lint fixes iliani8 JY-20725-harlle-HS-search-rate-limit* (@6 A 110 28Windovw Clienttest.oho (Workina Tree M17 Client.php (Working Tree) 9, M Xm MatchActivitvermData.pho (Workina Tree Mapp > Services > Crm > Hubspot › « Client.php > a Client › © getRateLimitCacheKey()class Client extends Baseclient 1mplements HubspotCllentintertaceВНВВВВВВВВВВВР5private tunction executeRequest callable sapicallwarning(' [Hubspot] Received 429 from API','config_id'→ $this->config->getId(),'retry_after' = $retryAfter,reason= Se->getmessage),throw new RateLimitExceptionHubspot returned 429. SretrvA1throw $e;pravate tunctzon getkateLzmzttachekeyl: strangreturn sprintf('hubspot:ratelimit:portal:&d, $this->config->getid(),public function isHubspotRateLimit(Throwable $e): boolif (Se instanceof BadRequestse instanceot DealAnizxcention11 Se instanceof \GuzzleHttol Excentionl RequestExcentionreturn (int) $e->getCode() === 429;public function parseRetryAfter(Throwable $e): intif (method_exists($e,Sheaders = Se->getResponseHeaders() ?: []:svalue = $headers ('Retry-After'] ?7 Sheaders I'retry-after'] 27 nl1t 1s array Svalue) <Svalue = svalue|01 22 null:ifis numeric(svaluereturn (int) svalueSmessage = strtolower Se->qetMessaqe0):if (str_contains ($message,'daily') ^return'ten secondly')) {if (str_contains ($message,"secondly"return 1:sthic_slon_swarnina/*[Hubsnotl No retrvcafter header or known meccaaeIR RateLimitExceptiOn.php (Workina Tree &MJ9MMclass Client extends Baseclient 1mplements Hubspotclientintertace10816911011111127142143|1441451private tunction executeRequest callable sapicall»warning(' [Hubspot) Received 429 from API', 1"config_id'→ sches->contto-getto,'retry_after' = $retryAfter,reason= Se->getmessage),throw new RateLimitexception( "Hubspot returned 429', SretryAfter,throw se.private tunction getRateLimitCachekey): stringreturn sprintf('hubspot:ratelimit:config:%d, $this->config->getid));private function isHubspotRateLimit(Throwable $e): boolif ($e instanceof BadRequestse instanceot DealAo1zxceotion11 Se instanceof \GuzzleHttol Excention|RequestExcentionreturn (int) $e->getCode() === 429;return talse;private function parseRetryAfter(Throwable $e): intif (method_exists($e,sheaders = se->geckesponseneaders ?: uJisvalue = $headers ['Retry-After'] ?? Sheaders ('retry-after'] 27 null;1t is array Svalue) ‹svalve = svalve 0 ? null.ifis numeric SvallueSmessage = strtolower(Se->qetMessace0):if (str_contains($message, 'daily')) {return 600;if (str_contains($message, 'ten secondly')) {if (str_contains($message, 'secondly')) {sthic_sloa_swarnina/:fHubsnotl No retrvcafter header or known$0 lil100% LzMon 11 May 17:56:20•2 C323 Review rate limit handli...Review rate limit handling for HubSpot j...Here are code reveiw suggestions: IssuesexpecteaMin' → 36,• Now run all affected tests to verify everything passes:Bash Run all affected teststests/Unit/Services/Crm/Hubspot/Pagination/HubspotPaginationServiceTest.php \olm Command.unnina inhackaround witth_T0.hhu9az8vk.Ontoutk.heine_waltitento./naivate//tmo//alaude_50W/2lkers-Ikas-Tminnv-ann//cae8117hd-3c48-4a63-h236_2f1d87Af2e4d/taskc/hhu9az8vk..om• Bash Check test outoutcat /private/tmp/claude-501/-Users-lukas-jiminny-app/cae817bd-3c48-4c63-b336-2f1d874f2e4d/tasks/bhu9az8vk.output 2>/dev/null | tail -30OUT PHPUnit 11.5.55 by Sebastian Bergmann and contributors.PHP 8.3.30• Thouaht for 2c )• All 120 tests pass. Here's a summary of every change made to address the review feedback:Issue 1 - Batch API calls bypass rate-limit handlingClient nho:?05-021-hatchRead0htleatc/Anow.wrans the ShatchConfintLaniWsread/luucalllinsideexeantteReatiest/lso./120c.from.batch.endnoints.are.cauaht.hvathe.same.circuitahreakerlodic.Adedicated catch (RateLimitException se) ( throw Se; } before the general catch ensures the rate-limit exception propagates to the job middleware instead of being swallowed by handleBatchError ()and re-wrapped as ermexceptzon ,Issue 2 - Misleading cache key nameClient.php:120 - Renamed hubspot:ratelimit:portal:%d → hubspot:ratelimit:config:&d . The key was already scoped to the config record ID (not a HubSpot portal ID); the name now matches theintent. The correspondina strina in CuientTest.oho was vodated to match.Issue 3 - isHubspotRateLimit() and parseRetryAfter() visibilityCalllentanhow/kWky-IBoth methods changedIfrom.nuhGtto.navateNeitherlisinthelnterface.andinofhing.outsidelexccutcRemuest70calstthem.the direct&testsiin.dbientalesia.nhn_weretuodateditoIssue o — Test constants duplicated trom production codeWondloEmhenatPatalfmitTestnhn.Pemoved.the.three.nrivate.constants/IMAXRETRYDELAYDUIMTNRETRY_DEWAYOIUITER_SFCONDSI.andlinlinedIthe concretelboundary.values.with commentsie.c'exnectedMax" → 605.//600 + 51itterIfthe middleware constants chande.these tests willnow failratherthan.silently.nass against stale exnected valuesIccue 6 - Douhle-clamnina in Patol jni+EycontionRateLimitException.php - The clamping moved to the constructor ( $this->retryAfter :• max(1, SretryAfter) ). The property is no longer a promoted parameter (which would prevent assignment in theconstructor bodv. getRetrvAftero is now a simole return. The external behaviour is identical, confirmed by the existing test suiteIssue 7 - Non-obvious RateLimitException path in failed()MatchActivitvCrmData.oho:172 - Added a three-line coent explainina that RateLimitExcention oniv reaches failedo when retrvuntilo exoires while the iob is still held in the released state.because the middleware calls release() rather than re-throwing.9 Ece to focuc or unfocuc Claude+00' Client.oho#Ask before edit.0 Not Committed Yet Ln 120, Col 1Spaces: 4 UTF-8 LF PHP 8 Signin 8.3 g...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20301
|
|
20302
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20302
|
|
20303
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20303
|
|
20304
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard
delay = max(1, min(10, 600)) + random_int(0, 5) → 10 to 15 seconds
So all 10 jobs land back in the queue, scheduled to fire between t=10 and t=15.
The Core Problem — TTL Expires Before Released Jobs Fire
The Core Problem — TTL Expires Before Released Jobs Fire
The Redis key was set with
TTL = retryAfter = 10
. The jobs were released with
delay = retryAfter + jitter = 10–15
.
Copy code to clipboard
Redis key expires: t = 10
Jobs start firing: t = 10 to t = 15
These are nearly the same window. The very first job that fires at t=10 may race against the TTL expiry. Jobs firing at t=11–15
will find an empty cache
because the key already expired. So the circuit breaker provides
zero protection
for the second wave of the same jobs. They all miss the cache again and charge straight at HubSpot.
This is the core mismatch: the release delay was derived from the same value as the TTL, so they expire together. The circuit breaker is only effective for
new, independent jobs
that arrive during the t=0–10 window, not for the 10 jobs that triggered the 429 in the first place.
Issue 1 Compounded — Stale Delay Makes It Worse
Issue 1 Compounded — Stale Delay Makes It Worse
Say a new job (job #11) arrives at t=5. It reads the cache and gets
"10"
. The middleware releases it for 10 seconds → fires at t=15. But the rate limit ended at t=10. It waited 5 extra seconds for nothing, and it fires exactly when the 10 original jobs are also firing (t=10–15). So instead of smoothing the load, the stale delay actually
bunches job #11 into the same second wave
.
With the absolute-timestamp fix:
t=5: job #11 reads cache, computes remaining =
(0 + 10) - 5 = 5
→ released for 5 seconds → fires at t=10. Hits HubSpot right when the limit clears. Better.
Issue 2 — The Second Wave Is the Same Thundering Herd
Issue 2 — The Second Wave Is the Same Thundering Herd
After the release, jitter gives a 5-second spread across 10 workers. But jitter only randomises
within each job
— it doesn't coordinate across workers. So the second wave looks like:
Copy code to clipboard
t=10: jobs 1, 2 fire → cache expired → hit HubSpot
t=11: jobs 3, 4 fire → cache expired → hit HubSpot
t=12: jobs 5, 6 fire → cache expired → hit HubSpot
...
If HubSpot is still rate-limiting at t=10 (which it may be — you got a 429 because you exceeded the burst limit, not necessarily the per-second limit), this produces
another wave of 429s
with a nearly identical delay spread, repeating the pattern. Each cycle chews through the
retryUntil()
window.
retryUntil() exhaustion
retryUntil()
exhaustion
With
retryAfter = 600
(daily limit, parsed as 600s) and
RETRY_WINDOW_MINUTES = 30
(1800s):
Copy code to clipboard
Wave 1: t=0 → 429, release for 600–605s
Wave 2: t=600 → 429 (HubSpot still limiting), release for 600–605s
Wave 3: t=1200 → 429, release for 600–605s → scheduled at t=1800
t=1800 = retryUntil() deadline → job permanently failed
Three strikes and the job is dead. For a spike of 10 workers all doing this in lockstep, you could permanently fail all 10 within 30 minutes with just two failed attempts each.
What Actually Helps vs What Doesn't
What Actually Helps vs What Doesn't
Mechanism
Intended purpose
Does it help in this scenario?
Redis circuit breaker
Block jobs while rate limit is active
Only helps jobs that arrive
after
the first 429 is cached. Doesn't help the 10 original jobs on their second attempt because the TTL has expired.
random_int(0, JITTER_SECONDS)
Prevent thundering herd
Spreads 10 jobs over 5 seconds — helpful but insufficient at scale
retryUntil()
/
maxExceptions
Bound total retries
retryUntil()
can be exhausted by rate-limit releases;
maxExceptions
doesn't count rate limit releases
Mechanism
Redis circuit breaker
random_int(0, JITTER_SECONDS)
retryUntil()
/
maxExceptions
Intended purpose
Block jobs while rate limit is active
Prevent thundering herd
Bound total retries
Does it help in this scenario?
Only helps jobs that arrive
after
the first 429 is cached. Doesn't help the 10 original jobs on their second attempt because the TTL has expired....
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20304
|
|
20305
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard
delay = max(1, min(10, 600)) + random_int(0, 5) → 10 to 15 seconds
So all 10 jobs land back in the queue, scheduled to fire between t=10 and t=15.
The Core Problem — TTL Expires Before Released Jobs Fire
The Core Problem — TTL Expires Before Released Jobs Fire
The Redis key was set with
TTL = retryAfter = 10...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20305
|
|
20306
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20306
|
|
20307
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard
delay = max(1, min(10, 600)) + random_int(0, 5) → 10 to 15 seconds
So all 10 jobs land back in the queue, scheduled to fire between t=10 and t=15.
The Core Problem — TTL Expires Before Released Jobs Fire
The Core Problem — TTL Expires Before Released Jobs Fire
The Redis key was set with
TTL = retryAfter = 10
. The jobs were released with
delay = retryAfter + jitter = 10–15
.
Copy code to clipboard
Redis key expires: t = 10
Jobs start firing: t = 10 to t = 15
These are nearly the same window. The very first job that fires at t=10 may race against the TTL expiry. Jobs firing at t=11–15
will find an empty cache
because the key already expired. So the circuit breaker provides
zero protection
for the second wave of the same jobs. They all miss the cache again and charge straight at HubSpot.
This is the core mismatch: the release delay was derived from the same value as the TTL, so they expire together. The circuit breaker is only effective for
new, independent jobs
that arrive during the t=0–10 window, not for the 10 jobs that triggered the 429 in the first place.
Issue 1 Compounded — Stale Delay Makes It Worse
Issue 1 Compounded — Stale Delay Makes It Worse
Say a new job (job #11) arrives at t=5. It reads the cache and gets
"10"
. The middleware releases it for 10 seconds → fires at t=15. But the rate limit ended at t=10. It waited 5 extra seconds for nothing, and it fires exactly when the 10 original jobs are also firing (t=10–15). So instead of smoothing the load, the stale delay actually
bunches job #11 into the same second wave
.
With the absolute-timestamp fix:...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20307
|
|
20308
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard
delay = max(1, min(10, 600)) + random_int(0, 5) → 10 to 15 seconds
So all 10 jobs land back in the queue, scheduled to fire between t=10 and t=15.
The Core Problem — TTL Expires Before Released Jobs Fire
The Core Problem — TTL Expires Before Released Jobs Fire
The Redis key was set with
TTL = retryAfter = 10
. The jobs were released with
delay = retryAfter + jitter = 10–15
.
Copy code to clipboard
Redis key expires: t = 10
Jobs start firing: t = 10 to t = 15
These are nearly the same window. The very first job that fires at t=10 may race against the TTL expiry. Jobs firing at t=11–15
will find an empty cache
because the key already expired. So the circuit breaker provides
zero protection
for the second wave of the same jobs. They all miss the cache again and charge straight at HubSpot.
This is the core mismatch: the release delay was derived from the same value as the TTL, so they expire together. The circuit breaker is only effective for
new, independent jobs
that arrive during the t=0–10 window, not for the 10 jobs that triggered the 429 in the first place.
Issue 1 Compounded — Stale Delay Makes It Worse
Issue 1 Compounded — Stale Delay Makes It Worse
Say a new job (job #11) arrives at t=5. It reads the cache and gets
"10"
. The middleware releases it for 10 seconds → fires at t=15. But the rate limit ended at t=10. It waited 5 extra seconds for nothing, and it fires exactly when the 10 original jobs are also firing (t=10–15). So instead of smoothing the load, the stale delay actually
bunches job #11 into the same second wave
.
With the absolute-timestamp fix:
t=5: job #11 reads cache, computes remaining =
(0 + 10) - 5 = 5
→ released for 5 seconds → fires at t=10. Hits HubSpot right when the limit clears. Better.
Issue 2 — The Second Wave Is the Same Thundering Herd
Issue 2 — The Second Wave Is the Same Thundering Herd
After the release, jitter gives a 5-second spread across 10 workers. But jitter only randomises
within each job
— it doesn't coordinate across workers. So the second wave looks like:
Copy code to clipboard
t=10: jobs 1, 2 fire → cache expired → hit HubSpot
t=11: jobs 3, 4 fire → cache expired → hit HubSpot
t=12: jobs 5, 6 fire → cache expired → hit HubSpot
...
If HubSpot is still rate-limiting at t=10 (which it may be — you got a 429 because you exceeded the burst limit, not necessarily the per-second limit), this produces
another wave of 429s
with a nearly identical delay spread, repeating the pattern. Each cycle chews through the
retryUntil()
window.
retryUntil() exhaustion
retryUntil()
exhaustion
With
retryAfter = 600
(daily limit, parsed as 600s) and
RETRY_WINDOW_MINUTES = 30
(1800s):
Copy code to clipboard
Wave 1: t=0 → 429, release for 600–605s
Wave 2: t=600 → 429 (HubSpot still limiting), release for 600–605s
Wave 3: t=1200 → 429, release for 600–605s → scheduled at t=1800
t=1800 = retryUntil() deadline → job permanently failed
Three strikes and the job is dead. For a spike of 10 workers all doing this in lockstep, you could permanently fail all 10 within 30 minutes with just two failed attempts each....
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20308
|
|
20309
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20309
|
|
20310
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard
delay = max(1, min(10, 600)) + random_int(0, 5) → 10 to 15 seconds
So all 10 jobs land back in the queue, scheduled to fire between t=10 and t=15.
The Core Problem — TTL Expires Before Released Jobs Fire
The Core Problem — TTL Expires Before Released Jobs Fire
The Redis key was set with
TTL = retryAfter = 10
. The jobs were released with
delay = retryAfter + jitter = 10–15
.
Copy code to clipboard
Redis key expires: t = 10
Jobs start firing: t = 10 to t = 15
These are nearly the same window. The very first job that fires at t=10 may race against the TTL expiry. Jobs firing at t=11–15
will find an empty cache
because the key already expired. So the circuit breaker provides
zero protection
for the second wave of the same jobs. They all miss the cache again and charge straight at HubSpot.
This is the core mismatch: the release delay was derived from the same value as the TTL, so they expire together. The circuit breaker is only effective for
new, independent jobs
that arrive during the t=0–10 window, not for the 10 jobs that triggered the 429 in the first place.
Issue 1 Compounded — Stale Delay Makes It Worse
Issue 1 Compounded — Stale Delay Makes It Worse
Say a new job (job #11) arrives at t=5. It reads the cache and gets
"10"
. The middleware releases it for 10 seconds → fires at t=15. But the rate limit ended at t=10. It waited 5 extra seconds for nothing, and it fires exactly when the 10 original jobs are also firing (t=10–15). So instead of smoothing the load, the stale delay actually
bunches job #11 into the same second wave
.
With the absolute-timestamp fix:
t=5: job #11 reads cache, computes remaining =
(0 + 10) - 5 = 5
→ released for 5 seconds → fires at t=10. Hits HubSpot right when the limit clears. Better.
Issue 2 — The Second Wave Is the Same Thundering Herd
Issue 2 — The Second Wave Is the Same Thundering Herd
After the release, jitter gives a 5-second spread across 10 workers. But jitter only randomises
within each job
— it doesn't coordinate across workers. So the second wave looks like:
Copy code to clipboard
t=10: jobs 1, 2 fire → cache expired → hit HubSpot
t=11: jobs 3, 4 fire → cache expired → hit HubSpot
t=12: jobs 5, 6 fire → cache expired → hit HubSpot
...
If HubSpot is still rate-limiting at t=10 (which it may be — you got a 429 because you exceeded the burst limit, not necessarily the per-second limit), this produces
another wave of 429s
with a nearly identical delay spread, repeating the pattern. Each cycle chews through the
retryUntil()
window.
retryUntil() exhaustion
retryUntil()
exhaustion
With
retryAfter = 600
(daily limit, parsed as 600s) and
RETRY_WINDOW_MINUTES = 30
(1800s):
Copy code to clipboard
Wave 1: t=0 → 429, release for 600–605s
Wave 2: t=600 → 429 (HubSpot still limiting), release for 600–605s
Wave 3: t=1200 → 429, release for 600–605s → scheduled at t=1800
t=1800 = retryUntil() deadline → job permanently failed
Three strikes and the job is dead. For a spike of 10 workers all doing this in lockstep, you could permanently fail all 10 within 30 minutes with just two failed attempts each.
What Actually Helps vs What Doesn't
What Actually Helps vs What Doesn't...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20310
|
|
20311
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20311
|
|
20312
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard
delay = max(1, min(10, 600)) + random_int(0, 5) → 10 to 15 seconds
So all 10 jobs land back in the queue, scheduled to fire between t=10 and t=15.
The Core Problem — TTL Expires Before Released Jobs Fire
The Core Problem — TTL Expires Before Released Jobs Fire
The Redis key was set with
TTL = retryAfter = 10
. The jobs were released with
delay = retryAfter + jitter = 10–15
.
Copy code to clipboard
Redis key expires: t = 10
Jobs start firing: t = 10 to t = 15
These are nearly the same window. The very first job that fires at t=10 may race against the TTL expiry. Jobs firing at t=11–15
will find an empty cache
because the key already expired. So the circuit breaker provides
zero protection
for the second wave of the same jobs. They all miss the cache again and charge straight at HubSpot.
This is the core mismatch: the release delay was derived from the same value as the TTL, so they expire together. The circuit breaker is only effective for
new, independent jobs
that arrive during the t=0–10 window, not for the 10 jobs that triggered the 429 in the first place.
Issue 1 Compounded — Stale Delay Makes It Worse
Issue 1 Compounded — Stale Delay Makes It Worse...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20312
|
|
20313
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G)
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20313
|
|
20314
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20314
|
|
20315
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20
HandleHubspotRateLimit.php:20
Copy code to clipboard
public function handle(object $job, callable $next): void
Calling
$job->attempts()
and
$job->release()
on
object
will work at runtime since Laravel always passes jobs that use
InteractsWithQueue
, but PHPStan/psalm won't verify this and it could silently break in tests with bare mocks (as your test does use a
Job
mock that has
attempts()
— so the test is fine, but the real type contract is invisible). Minor, but worth tightening to
\Illuminate\Contracts\Queue\Job
.
What works well ✓
What works well ✓
RateLimitException
is caught by the middleware
before
it increments
maxExceptions
, so rate-limit releases don't consume the 3-exception budget for real failures.
The middleware's jitter (
random_int(0, 5)
) prevents thundering herd on release.
The log-sampling logic (
attempts <= 3 || attempts % 10 === 0
) keeps logs useful without flooding.
ShouldBeUnique
+
uniqueFor()
correctly prevents duplicate jobs queuing up during the rate-limit wait window.
RateLimitException
propagates cleanly through the DB transaction (rolled back), then through the job catch block (logged conditionally), then to the middleware which handles it.
The pagination service re-throws
RateLimitException
as-is and resets pagination state on the next job attempt — safe because offset restarts from 0.
Message actions
Expand on 1 and 2. Let's consider there are several jobs running from ten workers. They are out fired instantly, and therefore there is no cache when they look for cache, but once they receive 429, they store cache. Can there be some potential issues with that flow?
Thought for 91s
Thought for 91s
Let me trace the exact race conditions step by step.
Wave 1 — The Initial Burst (t=0)
Wave 1 — The Initial Burst (t=0)
All 10 workers execute simultaneously. Each does:
Copy code to clipboard
$cachedRetryAfter = Redis::get($cacheKey); // ALL TEN: null — cache miss
Cache doesn't exist yet, so all 10 proceed and hammer HubSpot's search endpoint simultaneously. HubSpot's search limit is 5 RPS per portal — 10 concurrent requests guarantees 429s. All 10 get 429, all 10 run:
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter); // e.g. TTL=10, value="10"
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
All 10 write the same key at nearly the same millisecond. Redis just accepts last-write-wins — no error, no coordination, 10 redundant writes. Then all 10 are caught by the middleware and released:
Copy code to clipboard
delay = max(1, min(10, 600)) + random_int(0, 5) → 10 to 15 seconds
So all 10 jobs land back in the queue, scheduled to fire between t=10 and t=15.
The Core Problem — TTL Expires Before Released Jobs Fire
The Core Problem — TTL Expires Before Released Jobs Fire
The Redis key was set with
TTL = retryAfter = 10
. The jobs were released with
delay = retryAfter + jitter = 10–15
.
Copy code to clipboard
Redis key expires: t = 10
Jobs start firing: t = 10 to t = 15
These are nearly the same window. The very first job that fires at t=10 may race against the TTL expiry. Jobs firing at t=11–15
will find an empty cache
because the key already expired. So the circuit breaker provides
zero protection
for the second wave of the same jobs. They all miss the cache again and charge straight at HubSpot.
This is the core mismatch: the release delay was derived from the same value as the TTL, so they expire together. The circuit breaker is only effective for...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20315
|
|
20316
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20316
|
|
20317
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G)
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
iTerm2ShellEditViewSessionScriptsProfilesWindowHelpAPP (-zsh)-zsh++DOCKER-232,9 +235,11₴81DEV (docker)₴2APP (-zsh)84* eparam string SobjectType The object type ('deals', 'companies', 'contacts', 'calls')* @param array<string, mixed› Spayload The search payload with filters, sorts, properties, etc.ereturn array Thesearch response with'results'total', 'paging'keys* @throws RateLimitException When rate limit is hit* @throws HubspotException On API errors* @return array The search response with 'results', 'total','paging' keyspublic function search(string SobjectType, array Spayload): array-zsh>0 ll*5screenpipe"100% <78• Mon 11 May 18:15:51181€ 886-zsh*7 |+APPend diff4) app/Console/Commands/JiminnyDebugCommand.php (statement_indentation)begin diff--- /home/jiminny/app/Console/Commands/JiminnyDebugCommand.php+++/home/jiminny/app/Console/Commands/JiminnyDebugCommand.php-359,11 +359,11 @ScrmService = ScrmResolver->prepareCrmService();-//-//for (Si = 0; si < 3; Si++) {if (Si%25 ===0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {$i}");-//-//.ScrmService->syncOpportunity('374720564');if ($i % 25 === 0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {Si}");////ScrmService-›syncOpportunity('374720564');ScrmService-›matchByName('Robot');end diffFixed 4 of 5666 files in 146.870 seconds, 60.00 MB memory usedWhat's next:Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug docker_lamp_1...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20317
|
|
20318
|
*selectionWindovSOURCE CONTROw Clienttest.oho (Wor *selectionWindovSOURCE CONTROw Clienttest.oho (Workina Tree M17 Client.php (Working Tree) 9, M Xm MatchActivitvermData.pho (Workina Tree Mi RateLimitException.oho (Workina Tree)V CHANGESapp > Services > Crm > Hubspot › « Client.php > a Client › © getRateLimitCacheKey()class Client extends Baseclient 1mplements HubspotCllentintertacey Changes• .env.local• .env.nikilocal@ .env.otherE ids.txtraw sal_query.sq* WEBHOOK_FILTERING_IMPLEMENT...# JiminnyDebugCommand.php app/Cl** SimulateWebhooksCommand.php a..." CreateMockAskJiminnvReportResult..1* RateLimitException.php app/Except...w MatchActivitvCrmData.oho app/Jobs... M1* Client.php app/Services... U 9 + 9, Moadina.nno confid* favicon.ico public** Handle-lubsootRateLimitTest.oho t...** CanAccessAiReportsTest.php tests/...** ClientTest oho tests/Unit/Services/Cr.V GRAPHAuto O y vTO JY-20725 add ...• JY-20725 add tests Lukas Kovali!•JY-20725 add HS rate limit handling on a..Merae oulll request #12.Merae branch 'master' into .IY-20818..Merae oull reauest #12057 from limin….JY.20818 move ask liminnv renorts to..Merge pull request #12052 from jimin...Morao hranch Imactorl into.IV.20917Merge pull request #12047 from jimin...A IV.20662 roturn woid flier• JY-20662 Removed word_boost para...JY-20817: Fix deleting old tracks mih...Merge pull request #12043 from jimin...Merge branch 'master' into JY-19938.Merge pull request #12044 from Jimin…y 10000 Addod cancont informatinnJy-19938 reorder arrav mao ilianJY-19938: Address PR review commeprivate tunction executeRequest callable Sapicallsthis->log->warning(' [Hubspot] Received 429 from API', I"conT19.10= Sch1s→>cont10→>gec.d),'retry after' => SretrvAtter.throw new RateLimitException('Hubspot returned 429', $retryAfprivate function getRateLimitCachekey(): stringreturn sprintf('hubspot:ratelimit:portal:%d', sthis->config->getId())public tunction 1sHubspotRateL1m1t Throwable se: boolif Se instanceof BadRequest11 se instanceof DealAniExcention1l se inctancenf ContactAniFycentionIl se instanceof \GuzzleHttp\Exception\RequestExceptionreturn (1nt se->qetcode === 429:return false.public function parseRetryAfter(Throwable Se): intif (method_exists($e,'getResponseheaders')Sheaders = se->aetResoonsereaderso7:ASvalue = Sheaders ('Retrv-After'] 2? Sheaders ('retry-after'] 22 nlSifis arravsvallue)Svalue = svaluel0l 22 nulleif (ic numeric(<valne)) !Smessane = strtolower(Se-saetMessade0)):if (str_contains($message,'daily')) {1t str contains smessage.'ten secondly'"){if (str contains(Smessage, "secondly') ^return 1Sthis->loq->warning(* (Hubspot) No retry-after header or known messageJY-19938 lint fixes ilian?8 JY-20725-harlle-HS-search-rate-limit* (@6 A 110 28class Client extends Baseclient 1mplements Hubspotclientintertace16b107122123-124112511401141|142|143144157|158150|160sthis->log->warning(' [Hubspot] Received 429 from API', ILean1o→ sches→>contlo-ream10,'config id'= Sch1s→>cont1g→>getla0),retry after' = SretryAtter.= Se->getMessageonthrow new RateLimitException('Hubspot returned 429', SretryAfter,throw se:private function getRateLimitCacheKey(): stringreturh sprinut nuospot: rateuimic: contig.0, schis→>contig-gecto,orivate tunction 1sHubspotRateL1m1t Throwable se: boouif Se instanceof BadRequest11 Se instanceof DealAniExcention11 ce inctancenf ContactAniFycentionreturn (1nt se->getcode === 429:return false;nrivate function narceRetrvAfter(Throwahle &e)• intif (method_exists(se,'getResponseHeaders")) €sheaders = se->aetResoonsereadersoiSvalue = Sheaders" "Retrv-After" ?? Sheaders "'retrv-after' 72 null.sfais arrav(svalue)4svalue = svalue al 22 nulleif (is numeric(Svalue)) {Smessage = strtolower(Se-saetMessage0):if (str_contains($message, 'daily')) ‹recurn b00.1t str contains Smessage, "ten secondlv') ‹str contains(Smessade. "secondlv')<return 1.Sthis->loq->warning('(Hubspot) No retry-after header or known message, us:$0 lil100% L2Mon 11 May 18:15:51• 2 C23 Review rate limit handli..Review rate limit handling for HubSpot jo...Here are code reveiw suggestions: Issues'expecteaMin' → 36,• Now run all affected tests to verify everything passes:Bash Run all affected teststests/Unit/Services/Crm/Hubspot/Pagination/HubspotPaginationServiceTest.php \nlCommand.aunniingin_hackaround witth_T0hhugaz8vk.Ontoutk.heina_waltten tor/naivate//tmo//alaude-501V/2lsersmkas-Tminnv-ann//cae8117bd-3c48-4c62_h226-2fa1d87Af2e4d/Ztasks/hu9azRvk..om• Bash Check test outoutcat /private/tmp/claude-501/-Users-lukas-jiminny-app/cae817bd-3c48-4c63-b336-2f1d874f2e4d/tasks/bhu9az8vk.output 2>/dev/null | tail -30OUT PHPUnit 11.5.55 by Sebastian Bergmann and contributors.PHP 8.3.30• Thouaht for 2c )• All 120 tests pass. Here's a summary of every change made to address the review feedback:Issue 1 - Batch API calls bypass rate-limit handlingClient nho:?05-021-hatchRead0htleatc/Anow.wrans the ShatchConfintLaniWsread/luucalllinsideexeantteReatiest/lso./120c.from.batch.endnoints.are.cauaht.hvathe.same.circuitahreakerlodicAdedicated catch (RateLimitException se) ( throw Se; } before the general catch ensures the rate-limit exception propagates to the job middleware instead of being swallowed by handleBatchError ()and re-wrapped as ermexceptzon .Issue 2 - Misleading cache key nameClient.php:120 - Renamed hubspot:ratelimit:portal:%d → hubspot:ratelimit:config:&d . The key was already scoped to the config record ID (not a HubSpot portal ID); the name now matches theintent. The correspondina strina in CuientTest.oho was vodated to match.Issue 3 - isHubspotRateLimit() and parseRetryAfter() visibilityCalllentanhow/kWky-IBoth methods changedIfrom.nuhGtto.navateNeitherlisinthelnterface.andinofhing.outsidelexccutcRemuest70calstthem.the direct&testsiin.dbientalesia.nhn_weretuodateditoIssue o — Test constants duplicated trom production codeWondloEmhenatPatalfmitTestnhn.Pemoved.the.three.nrivate.constants/IMAXRETRYDELAYDUIMTNRETRY_DEWAYOIUITER_SFCONDSI.andlinlinedIthe concretelboundary.values.with commentsie.g'expectedMax' => 605, // 600 + 5 jitter ). If the middleware constants change, these tests will now fail rather than silently pass against stale expected valuesIccue 6 - Douhle-clamnina in Patol Ini+EycontionRateLimitException.php - The clamping moved to the constructor ( $this->retryAfter =• max(1, SretryAfter) ). The property is no longer a promoted parameter (which would prevent assignment in theconstructor bodv. getRetrvAftero is now a simole return. The external behaviour is identical, confirmed by the existing test suiteIssue 7 - Non-obvious RateLimitException path in failed()MatchActivitvCrmData.oho:172 - Added a three-line coent explainina that RateLimitExcention oniv reaches failedo when retrvuntilo exoires while the iob is still held in the released state.because the middleware calls release() rather than re-throwing.9 Ece to focuc or unfocuc Claude+00' Client.ohoAsk before edit.0 Not Committed Yet Ln 120, Col 1Spaces: 4 UTF-8 LF PHP 8 Signin 8.3 g...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20318
|
|
20319
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
iTerm2ShellEditViewSessionScriptsProfilesWindowHelpAPP (-zsh)-zsh++DOCKER-232,9 +235,11 @@₴81DEV (docker)₴2APP (-zsh)84* eparam string SobjectType The object type ('deals', 'companies', 'contacts', 'calls')* @param array<string, mixed› Spayload The search payload with filters, sorts, properties, etc.ereturn array Thesearch response with'results'total', 'paging'keys* @throws RateLimitException When rate limit is hit* @throws HubspotException On API errors* @return array The search response with 'results', 'total','paging' keyspublic function search(string SobjectType, array Spayload): array-zsh>0 ll*5screenpipe"100% <78• Mon 11 May 18:15:54T₴1€ 886-zsh*7 |+APPend diff4) app/Console/Commands/JiminnyDebugCommand.php (statement_indentation)begin diff--- /home/jiminny/app/Console/Commands/JiminnyDebugCommand.php+++/home/jiminny/app/Console/Commands/JiminnyDebugCommand.php-359,11 +359,11 @ScrmService = ScrmResolver->prepareCrmService();-//-//for (Si = 0; si < 3; Si++) {if (Si%25 ===0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {$i}");-//-//.ScrmService->syncOpportunity('374720564');if ($i % 25 === 0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {Si}");////ScrmService-›syncOpportunity('374720564');ScrmService-›matchByName('Robot');end diffFixed 4 of 5666 files in 146.870 seconds, 60.00 MB memory usedWhat's next:Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug docker_lamp_1...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20319
|
|
20320
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
selectionVIeWSOURCE CONTROCHANGESMessage aenter to commit on "J.v Changes@ .env.local• .env.nikilocal@ .env.otherE ids.txtraw sal_query.sq* WEBHOOK_FILTERING IMPLEMENT.# JiminnyDebugCommand.php app/Cl"# SimulateWebhooksCommand.php a." CreateMockAskJiminnvReportResult..« RateLimitException.php app/Excepti..w MatchActivitvCrmData.oho app/Jobs... M#R Client.php app/Services/... *9 b + 9. Moadina.nno confid* favicon.ico public** Handle-lubsootRateLimitTest.oho t...# CanAccessAiReportsTest.php tests/...** ClientTest oho tests/Unit/Services/Cr.Windov• (Working Tree) Mm Clientlest.pho Workina TreeM# Client.pho (Workina Tree) 9,Mapp > Services > Crm › Hubspot › f* Client.php › •g Client › [ batchReadObjects()class Client extends Baseclient 1mpLements Hubspotclientintertacepub lic tunction getopportunityByld string scrmid, array stields: array290"contacts')@param array<string> $crmIds Array of HubSpot object IDs (max 100)* oparan arrayssuring> stielas Array oт property namesTO Tetch* @return array<string, array> Array keyed by CRM ID with object data29%Generic batch read method for HubSpot objectsprivate function batchRead0bjects(string $objectType, array $crmIds, array $fields): arrayif (empty(ScrmIds)) {return U:Sthis->validateBatchSize(Sobjecttype, Scrmids);Sthis-sensureValidTokenO:try {ShatchGonfin - Sthic-wcrostoRotchGonfiauration/cohioctTung).SbatchReadRequest = Sthis->prepareBatchRequest(SbatchConfig, ScrmIds, Sfields):Sresponse = sthis->executeRequestfn = Sbatchconfial'api'->readSbatchReadRequest:Sthis-svalidateAniResnonse/Sresnonse. SohiectTvne).Sresults = $this->processApiResults($response):suhis→>logbatchkesults(sobjecclype,scrmios, sresults;return Sresults;& catch (RateLimitException se) ‹throw se:catch (Throwable Se) 4$this->handleBatchError($e, $objectType, $crmIds):S0 l100% L2Mon 11 May 18:15:560 000V GRAPH{9 Autol÷O JY-20725 add m.• JY-20725 add tests Lukas Kovalik• JY-20725 add HS rate limit handling on a..Merae oulll request #12.Merae branch 'master' into .IY-20818.Merae oull reauest #12057 from limin….JY.20818 move ask liminnv renorts to..Merae oull reauest #12052 from limin…..Morao hranch Imactorl into.IV.20917Morao null roauoct #12047 from liminO 1y.20662 roturn void tlior331• JY-20662 Removed word boost para...JY-20817: Fix deleting old tracks mih..Merge pull request #12043 from jimin…Merge branch 'master' into JY-19938...Merge pull request #12044 from fimin…y 10000 Addod cancont informatinnJy-19938 reorder arrav mao ilianJY-19938: Address PR review commeJY-19938 lint fixes ilianiô JY-20725-harlle-HS-search-rate-limit* O@ 6 A 11 © 28nrivate function validateßatchSize(strina SobiectTvne, arrav scrmids)• voidi€ (count (Ccrmtde) > 100) ₫throw new \InvalidArgumentException("Batch size cannot exceed 100 {SobjectType}"):private function createBatchConfiguration(string SobiectType): arraySconfiqurations =batchReadRequest' = new \HubSpot\GLient\Grm\Deals\Mode1\BatchReadInoutSimolePublic0biectId().\HubSnot\Glient\Grm\Deals\Model|SimolePublic0biectId::class.'api' = $this->getNewInstance()->crm()->deals()->batchApi(),batchReadReguest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublic0biectId().'Inoutclass' => Huosoot Cuent Crm comoanes Mode Simo lepuoacoblectd::class• 2 CClient.php 9. M X# MatchActivitvermData.oho (Workina Tree MMRiDY-3 Review rate limit handli.Review rate limit handling for HubSpot jo..Here are code reveiw suggestions: Issues'expecteaMin' = 36,• Now run all affected tests to verify everything passes:• Bash Run all affected teststests/Unit/Services/Crm/Hubsoot/Paqination/HubsootPaqinationServiceTest.ohonlCommand.aunniingin_hackaround witth_T0hhugaz8vk.Ontoutk.heina_waltten tor/naivate//tmo//alaude-501V/2lsersmkas-Tminnv-ann//cae8117bd-3c48-4c62_h226-2fa1d87Af2e4d/Ztasks/hu9azRvk..om• Bash Check test outoutcat /private/tmp/claude-501/-Users-lukas-jiminny-app/cae817bd-3c48-4c63-b336-2f1d874f2e4d/tasks/bhu9az8vk.output 2>/dev/null | tail -30OUT PHPUnit 11.5.55 by Sebastian Bergmann and contributors.PHP 8.3.30Thouaht for 2c)All 120 tests pass. Here's a summary of every change made to address the review feedback:Issue 1 - Batch API calls bypass rate-limit handlingClient nho:?05-021-hatchRead0htleatc/Anow.wrans the ShatchConfintLaniWsread/luucalllinsideexeantteReatiest/lso./120c.from.batch.endnoints.are.cauaht.hvathe.same.circuitahreakerlodicAdedicated catch (RateLimitException Se) { throw se: } before the general catch ensures the rate-limit exception propagates to the iob middleware instead of being swallowed by handleBatchError()and re-wrapped as ermexceptzon ,Issue 2 - Misleading cache key nameClient.php:120 - Renamed hubspot:ratelimit:portal:%d → hubspot:ratelimit:confiq:%d. The key was already scoped to the config record ID (not a HubSpot portal ID); the name now matches theintent. The correspondina strina in CuientTest. oho was vodated to match.Issue 3 - isHubspotRateLimit() and parseRetryAfter() visibilityCalllentanhow/kWky-IBoth methods changedIfrom.nuhGtto.navateNeitherlisinthelnterface.andinofhing.outsidelexccutcRemuest70calstthem.the direct&testsiin.dbientalesia.nhn_weretuodateditoIssue o — Test constants duplicated trom production codeHandleHubspotRateLimitTest.php - Removed the three private constants (MAX RETRY DELAY, MIN RETRY DELAY. JITTER SECONDS ) and inlined the concrete boundary values with comments (e.a.'expectedMax' => 605, // 600 + 5 jitter ). If the middleware constants change, these tests will now fail rather than silently pass against stale expected valuesIccue 6 - Douhle-clamnina in Patol jni+EycontionRateLimitException.php - The clamping moved to the constructor ( $this->retryAfterconstructor bodv. getRetrvAftero is now a simole return. The external behaviour is identical, confirmed by the existing test suite= max(1, SretryAfter) ). The property is no longer a promoted parameter (which would prevent assignment in theIssue 7 - Non-obvious RateLimitException path in failed()MatchActivitvCrmData.oho:172 - Added a three-line coent explainina that RateLimitExcention oniv reaches failedo when retrvuntilo exoires while the iob is still held in the released state.because the middleware calls release() rather than re-throwing.9 Ece to focuc or unfocuc Claude' Client.ohoAsk before edit.Lukas Kovalik (5 months ago) Ln 307, Col 52wnonootUTF-8 LF [PHP 8 Signin 8.3P...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20320
|
|
20321
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
Editor Language Status: $(copilot) No inline suggestion available, Inline suggestions
LF
UTF-8
Spaces: 4
Ln 309, Col 40
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20321
|
|
20322
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
Editor Language Status: $(copilot) No inline suggestion available, Inline suggestions
LF
UTF-8
Spaces: 4
Ln 309, Col 40
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20322
|
|
20323
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
Editor Language Status: $(copilot) No inline suggestion available, Inline suggestions
LF
UTF-8
Spaces: 4
Ln 309, Col 39
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20323
|
|
20324
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
Editor Language Status: $(copilot) No inline suggestion available, Inline suggestions
LF
UTF-8
Spaces: 4
Ln 309, Col 39
git-commit Not Committed Yet
Not Committed Yet
Found 1 symbol in /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait....
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20324
|
|
20325
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
<?php
declare(strict_types=1);
namespace Jiminny\Services\Crm\Hubspot;
use HubSpot\Client\Crm\Deals\ApiException as DealApiException;
use HubSpot\Client\Crm\Contacts\ApiException as ContactApiException;
use HubSpot\Client\Crm\Companies\ApiException as CompanyApiException;
use HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectWithAssociations as ContactsWithAssociations;
use HubSpot\Client\Crm\Companies\Model\SimplePublicObjectWithAssociations as CompaniesWithAssociations;
use HubSpot\Client\Crm\Deals\Model\SimplePublicObjectWithAssociations as DealWithAssociations;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectInput;
use HubSpot\Client\Crm\Objects\Model\SimplePublicObjectWithAssociations as ObjectWithAssociations;
use HubSpot\Client\Crm\Pipelines\Model\Error;
use HubSpot\Client\Crm\Pipelines\Model\PipelineStage;
use HubSpot\Client\Crm\Properties\Model\Property;
use HubSpot\Discovery\Discovery;
use Jiminny\Exceptions\CrmException;
use Jiminny\Exceptions\RateLimitException;
use Jiminny\Exceptions\SocialAccountTokenInvalidException;
use Jiminny\Jobs\Crm\NoteObject;
use Jiminny\Models\Crm\Field;
use Jiminny\Services\Crm\BaseClient;
use Jiminny\Services\Crm\Hubspot\DTO\Response\Owner;
use Jiminny\Services\SocialAccountService;
use SevenShores\Hubspot\Exceptions\BadRequest;
use SevenShores\Hubspot\Exceptions\HubspotException;
use SevenShores\Hubspot\Factory;
use SevenShores\Hubspot\Http\Response;
use Jiminny\Services\Crm\Hubspot\Pagination\HubspotPaginationService;
use Illuminate\Support\Facades\Redis;
use Throwable;
/**
* @phpstan-type CrmFieldOption array{id:string, label:string, value?:string}
*/
class Client extends BaseClient implements HubspotClientInterface
{
public const string MIN_API_VERSION = '2';
public const string BASE_URL = '[URL_WITH_CREDENTIALS] T
*
* @param callable(): T $apiCall The API call to execute
*
* @throws RateLimitException When rate limit is hit or cached rate limit is active
*
* @return T The result of the API call
*/
private function executeRequest(callable $apiCall)
{
$cacheKey = $this->getRateLimitCacheKey();
$cachedExpiresAt = Redis::get($cacheKey);
if (is_string($cachedExpiresAt) && is_numeric($cachedExpiresAt)) {
$remaining = max(1, (int) $cachedExpiresAt - time());
throw new RateLimitException(
'Hubspot rate limit (cached circuit-breaker)',
$remaining,
);
}
try {
return $apiCall();
} catch (Throwable $e) {
if ($this->isHubspotRateLimit($e)) {
$retryAfter = $this->parseRetryAfter($e);
// NX: only the first job to receive a 429 in this burst sets the key.
// Subsequent 429s in the same burst leave the TTL untouched so the
// window is not reset by every concurrent job hitting the limit.
Redis::set($cacheKey, (string) (time() + $retryAfter), ['nx', 'ex' => $retryAfter]);
$this->log->warning('[Hubspot] Received 429 from API', [
'team_id' => $this->config->team_id,
'config_id' => $this->config->getId(),
'retry_after' => $retryAfter,
'reason' => $e->getMessage(),
]);
throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);
}
throw $e;
}
}
private function getRateLimitCacheKey(): string
{
return sprintf('hubspot:ratelimit:config:%d', $this->config->getId());
}
private function isHubspotRateLimit(Throwable $e): bool
{
if ($e instanceof BadRequest
|| $e instanceof DealApiException
|| $e instanceof ContactApiException
|| $e instanceof CompanyApiException
|| $e instanceof \GuzzleHttp\Exception\RequestException
) {
return (int) $e->getCode() === 429;
}
return false;
}
private function parseRetryAfter(Throwable $e): int
{
if (method_exists($e, 'getResponseHeaders')) {
$headers = $e->getResponseHeaders() ?: [];
$value = $headers['Retry-After'] ?? $headers['retry-after'] ?? null;
if (is_array($value)) {
$value = $value[0] ?? null;
}
if (is_numeric($value)) {
return (int) $value;
}
}
$message = strtolower($e->getMessage());
if (str_contains($message, 'daily')) {
return 600;
}
if (str_contains($message, 'ten secondly')) {
return 10;
}
if (str_contains($message, 'secondly')) {
return 1;
}
$this->log->warning('[Hubspot] No retry-after header or known message, using default', [
'exception_class' => get_class($e),
'message' => $message,
]);
return 10;
}
public function getMinimumApiVersion(): string
{
return self::MIN_API_VERSION;
}
public function getInstance(): Factory
{
return new Factory([
'key' => $this->accessToken,
'oauth2' => true,
'base_url' => $this->baseUrl,
]);
}
public function getNewInstance(): Discovery
{
return \HubSpot\Factory::createWithAccessToken($this->accessToken);
}
/**
* Secondly and daily limits for Hubspot API
*
* Product Tier: Free & Starter | Professional & Enterprise | API add-on (any tier)
* Burst: 100/10 seconds | 150/10 seconds | 200/10 seconds
* Daily: 250,000 | 500,000 | 1,000,000
*
* Official documentation states: The search endpoints are rate limited to five requests per second.
* Since with 5 RPS were still hitting secondly rate limits we lowered it to 4
*/
public function getPaginatedData(array $payload, string $type, int $offset = 0): array
{
$total = 0;
$lastId = null;
$rows = [];
foreach ($this->getPaginatedDataGenerator($payload, $type, $offset, $total, $lastId) as $row) {
$rows[] = $row;
}
return ['results' => $rows, 'total' => $total, 'last_record' => $lastId];
}
/**
* @throws HubspotException
* @throws SocialAccountTokenInvalidException
* @throws BadRequest
*/
public function getPaginatedDataGenerator(
array $payload,
string $type,
int $offset = 0,
int &$total = 0,
?string &$lastRecordId = null
): \Generator {
return $this->paginationService->getPaginatedDataGenerator(
$this,
$payload,
$type,
$offset,
$total,
$lastRecordId
);
}
/**
* Execute a search request against HubSpot CRM objects with rate limiting.
*
* @param string $objectType The object type ('deals', 'companies', 'contacts', 'calls')
* @param array<string, mixed> $payload The search payload with filters, sorts, properties, etc.
*
* @throws RateLimitException When rate limit is hit
* @throws HubspotException On API errors
*
* @return array The search response with 'results', 'total', 'paging' keys
*/
public function search(string $objectType, array $payload): array
{
$endpoint = self::BASE_URL . "/crm/v3/objects/{$objectType}/search";
return $this->executeRequest(function () use ($endpoint, $payload) {
$response = $this->getInstance()->getClient()->request('POST', $endpoint, ['json' => $payload]);
return $response->toArray();
});
}
/**
* @throws DealApiException
* @throws CrmException
*/
public function getOpportunityById(string $crmId, array $fields): array
{
try {
$deal = $this->getNewInstance()->crm()->deals()->basicApi()->getById(
$crmId,
implode(',', $fields),
'companies,contacts'
);
} catch (DealApiException $e) {
$this->log->info('[Hubspot] Failed to fetch opportunity', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $deal instanceof DealWithAssociations) {
throw new CrmException('Deal not found');
}
return [
'id' => $deal->getId(),
'properties' => $deal->getProperties(),
'associations' => $deal->getAssociations(),
];
}
/**
* Generic batch read method for HubSpot objects
*
* @param string $objectType The object type ('deals', 'companies', 'contacts')
* @param array<string> $crmIds Array of HubSpot object IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with object data
*/
private function batchReadObjects(string $objectType, array $crmIds, array $fields): array
{
if (empty($crmIds)) {
return [];
}
$this->validateBatchSize($objectType, $crmIds);
$this->ensureValidToken();
try {
$batchConfig = $this->createBatchConfiguration($objectType);
$batchReadRequest = $this->prepareBatchRequest($batchConfig, $crmIds, $fields);
$response = $this->executeRequest(fn () => $batchConfig['api']->read($batchReadRequest));
$this->validateApiResponse($response, $objectType);
$results = $this->processApiResults($response);
$this->logBatchResults($objectType, $crmIds, $results);
return $results;
} catch (RateLimitException $e) {
throw $e;
} catch (\Throwable $e) {
$this->handleBatchError($e, $objectType, $crmIds);
}
}
private function validateBatchSize(string $objectType, array $crmIds): void
{
if (count($crmIds) > 100) {
throw new \InvalidArgumentException("Batch size cannot exceed 100 {$objectType}");
}
}
private function createBatchConfiguration(string $objectType): array
{
$configurations = [
'deals' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Deals\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Deals\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->deals()->batchApi(),
],
'companies' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Companies\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Companies\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->companies()->batchApi(),
],
'contacts' => [
'batchReadRequest' => new \HubSpot\Client\Crm\Contacts\Model\BatchReadInputSimplePublicObjectId(),
'inputClass' => \HubSpot\Client\Crm\Contacts\Model\SimplePublicObjectId::class,
'api' => $this->getNewInstance()->crm()->contacts()->batchApi(),
],
];
if (! isset($configurations[$objectType])) {
throw new \InvalidArgumentException("Unsupported object type: {$objectType}");
}
return $configurations[$objectType];
}
private function prepareBatchRequest(array $batchConfig, array $crmIds, array $fields): object
{
$batchReadRequest = $batchConfig['batchReadRequest'];
$inputClass = $batchConfig['inputClass'];
$inputs = array_map(function ($crmId) use ($inputClass) {
$input = new $inputClass();
$input->setId($crmId);
return $input;
}, $crmIds);
$batchReadRequest->setInputs($inputs);
$batchReadRequest->setProperties($fields);
return $batchReadRequest;
}
private function validateApiResponse($response, string $objectType): void
{
if (! $response) {
throw new CrmException("HubSpot API returned null response for {$objectType} batch read");
}
}
private function processApiResults($response): array
{
$results = [];
$responseResults = $response->getResults();
if ($responseResults) {
foreach ($responseResults as $object) {
if ($object && $object->getId()) {
$results[$object->getId()] = [
'id' => $object->getId(),
'properties' => $object->getProperties() ?: [],
];
}
}
}
return $results;
}
private function logBatchResults(string $objectType, array $crmIds, array $results): void
{
$this->log->info("[HubSpot] Batch fetched {$objectType}", [
'requested_count' => count($crmIds),
'returned_count' => count($results),
'crm_ids' => $crmIds,
]);
}
private function handleBatchError(\Throwable $e, string $objectType, array $crmIds): void
{
$errorMessage = $e->getMessage() ?: 'Unknown error';
$errorTrace = $e->getTraceAsString() ?: 'No trace available';
$this->log->error("[HubSpot] Failed to batch fetch {$objectType}", [
'crm_ids' => $crmIds,
'error' => $errorMessage,
'trace' => $errorTrace,
]);
throw new CrmException("Failed to batch fetch {$objectType}: " . $errorMessage);
}
/**
* Batch read multiple opportunities by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot deal IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with opportunity data
*/
public function getOpportunitiesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('deals', $crmIds, $fields);
}
/**
* Batch read multiple companies by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot company IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with company data
*/
public function getCompaniesByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('companies', $crmIds, $fields);
}
/**
* Batch read multiple contacts by their CRM IDs
*
* @param array<string> $crmIds Array of HubSpot contact IDs (max 100)
* @param array<string> $fields Array of property names to fetch
*
* @return array<string, array> Array keyed by CRM ID with contact data
*/
public function getContactsByIds(array $crmIds, array $fields): array
{
return $this->batchReadObjects('contacts', $crmIds, $fields);
}
/**
* @throws CompanyApiException
* @throws CrmException
*/
public function getAccountById(string $crmId, array $fields): array
{
try {
$company = $this->getNewInstance()->crm()->companies()->basicApi()->getById(
$crmId,
implode(',', $fields),
);
} catch (CompanyApiException $e) {
$this->log->info('[Hubspot] Failed to fetch account', [
'crm_id' => $crmId,
'reason' => $e->getMessage(),
]);
throw $e;
}
if (! $company instanceof CompaniesWithAssociations) {
throw new CrmException('Account not found');
}
return [
'id' => $company->getId(),
'properties' => $company->getProperties(),
];
}
/**
* @throws ContactApiException
* @throws CrmException
*/
public function getContactById(string $crmId, array $fields): array
{
try {
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
Editor Language Status: $(copilot) No inline suggestion available, Inline suggestions
LF
UTF-8
Spaces: 4
Ln 309, Col 39
git-commit Not Committed Yet
Not Committed Yet
Found 1 symbol in /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20325
|
|
20326
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20326
|
|
20327
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
iTerm2ShellEditViewSessionScriptsProfilesWindowHelpAPP (-zsh)-zsh++DOCKER-232,9 +235,11 @@"-₴1DEV (docker)₴2APP (-zsh)84* eparam string SobjectType The object type ('deals', 'companies', 'contacts', 'calls')* @param array<string, mixed› Spayload The search payload with filters, sorts, properties, etc.ereturn array Thesearch response with'results'total', 'paging'keys* @throws RateLimitException When rate limit is hit* @throws HubspotException On API errors* @return array The search response with 'results', 'total','paging' keyspublic function search(string SobjectType, array Spayload): array-zsh>0 ll*5screenpipe"100% <78• Mon 11 May 18:16:45T₴1O ₴6-zsh*7 |+APPend diff4) app/Console/Commands/JiminnyDebugCommand.php (statement_indentation)begin diff--- /home/jiminny/app/Console/Commands/JiminnyDebugCommand.php+++/home/jiminny/app/Console/Commands/JiminnyDebugCommand.php-359,11 +359,11 @ScrmService = ScrmResolver->prepareCrmService();-//-//for (Si = 0; si < 3; Si++) {if (Si%25 ===0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {$i}");-//-//.ScrmService->syncOpportunity('374720564');if ($i % 25 === 0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {Si}");////ScrmService-›syncOpportunity('374720564');ScrmService-›matchByName('Robot');end diffFixed 4 of 5666 files in 146.870 seconds, 60.00 MB memory usedWhat's next:Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug docker_lamp_1...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20327
|
|
20328
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20328
|
|
20329
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20329
|
|
20330
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
iTerm2ShellEditViewSessionScriptsProfilesWindowHelpAPP (-zsh)-zsh++DOCKER-232,9 +235,11- 881DEV (docker)₴2APP (-zsh)84* eparam string SobjectType The object type ('deals', 'companies', 'contacts', 'calls')* @param array<string, mixed› Spayload The search payload with filters, sorts, properties, etc.ereturn array Thesearch response with'results'total', 'paging'keys* @throws RateLimitException When rate limit is hit* @throws HubspotException On API errors* @return array The search response with 'results', 'total','paging' keyspublic function search(string SobjectType, array Spayload): array-zsh>0 ll*5screenpipe"100% <78• Mon 11 May 18:17:02T₴1O ₴6-zsh*7 |+APPend diff4) app/Console/Commands/JiminnyDebugCommand.php (statement_indentation)begin diff--- /home/jiminny/app/Console/Commands/JiminnyDebugCommand.php+++/home/jiminny/app/Console/Commands/JiminnyDebugCommand.php-359,11 +359,11 @ScrmService = ScrmResolver->prepareCrmService();-//-//for (Si = 0; si < 3; Si++) {if (Si%25 ===0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {$i}");-//-//.ScrmService->syncOpportunity('374720564');if ($i % 25 === 0) {Sthis->info("Syncing opportunity {$1}");Sthis->info("Matching contact {Si}");////ScrmService-›syncOpportunity('374720564');ScrmService-›matchByName('Robot');end diffFixed 4 of 5666 files in 146.870 seconds, 60.00 MB memory usedWhat's next:Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug docker_lamp_1...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20330
|
|
20331
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
selectionvIewSOURCE CONTROCHANGESMessage aenter to commit on "J.v Changes@ .env.local• .env.nikilocal@ .env.otherE ids.txtraw sal_query.sq* WEBHOOK_FILTERING IMPLEMENT.# JiminnyDebugCommand.php app/Cl"# SimulateWebhooksCommand.php a." CreateMockAskJiminnvReportResult..« RateLimitException.php app/Excepti..wMatchActivitvCrmData.oho app/Jobs….. M#R Client.php app/Services/... *9 b + 9. Moadina.nno confid* favicon.ico public** Handle-lubsootRateLimitTest.oho t...# CanAccessAiReportsTest.php tests/...** ClientTest oho tests/Unit/Services/Cr.Sõ3V GRAPHP9Auto @ ÷÷ 1O JY-20725 add m.• JY-20725 add tests Lukas Kovali!• JY-20725 add HS rate limit handling on a..Merae oulll request #2.Merae branch 'master' into .IY-20818..Merae oull reauest #12057 from limin….JY.20818 move ask liminnv renorts to..Merae oull reauest #12052 from limin…..Morao hranch Imactorl into.IV.20917Morao null roauoct #12047 from liminA IV.20662 roturn woid flierA IY.20662 Pomoved word honct naraJY-20817: Fix deleting old tracks mih..Merge pull request #12043 from jimin…Merge branch 'master' into JY-19938.Merge pull request #12044 from Jimin…y 10000 Addod cancont informatinnIv.10029 reorder array map ilianJY-19938: Address PR review commeJY-19938 lint fixes ilianiô JY-20725-hardle-HS-search-rate-limit* (@6 A 11 © 28Windov• (Working Tree) Mm Clientlest.pho Workina TreeMClient.php (Working Tree) 9, M Xapp > Services > Crm › Hubspot › f* Client.php › •g Client › getRateLimitCacheKey()class Client extends Baseclient 1mplements HubspotCllentintertaceprivate tunction executeRequest callable Sapicall$this->log->warning('[Hubspot) Received 429 from API', [→ stnts→>contlg→rean10"conT19.10= Sch1s→>cont10→>get.a0),'retry after' => SretrvAtter.throw new RateLimitException('Hubspot returned 429', $retryAfter, $e);private function getRateLimitCacheKey(): stringreturn sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());public tunction 1sHubspotRateL1m1t Throwable se: boolif Se instanceof BadRequest11 se instanceof DealAniExcention11 se inctancenf ContactAniFycentionIl $e instanceof \GuzzleHttp\Exception\RequestExceptionreturn (1nt se->qetcode === 429:return false.public function parseRetryAfter(Throwable Se): intif (method exists(se'getResponseheaders')Sheaders = se->aetResoonsereaderso7:ASvalue = Sheaders ('Retrv-After' ?? Sheaders ""retry-after' ?? nuul:ifais arravsvallue)Svalue = svaluel0l 22 nulleif (ic numeric(<valne)) !return (int) $valueSmessane = strtolower(Se-saetMessade0)):if (str contains($message,"dailv'))4sif (str contains($message, 'ten secondlv')) {if (str contains(Smessaae, 'secondlv')) {return 1Sthis->loq->warning(* [Hubspot) No retry-after header or known message, using defaClient.php 9, M123-S0 l100% L2Mon 11 May 18:17:01# MatchActivitvermData.oho (Workina Tree Mclass Client extends Baseclient implements HubspotClientintertaceprivate tunction executeRequest callable sapicall$this->log->warning('[Hubspot) Received 429 from API', [Lean1o→ schis→>contlo→rean1o,config id= Schis→>cont1g→>getla),'retry atter' = SretryAtter= Se->getMessageon10 :throw new RateLimitException('Hubspot returned 429', SretryAfter, $e);throw se;private function getRateLimitCacheKey(): stringreturn sprintf('hubspot:ratelimit:config:%d', $this->config->getId());private tunction 1sHubspotRateL1m1t (Throwable se: boolif Se instanceof BadRequest11 Se instanceof DealAniExcentionllse inctancenf ContactAniFycentionSe instanceof \GuzzleHttp\Exception\RequestExceptionreturn (1nt Se->getcode === 429:return false:orivace runction parseretrvafter Throwahle so. intif (method exists(Se'getResponseHeaders') ) {sheaders = se->oetResoonsereaderso?:aSvalue = Sheaders ""Retrv-After' ?? Sheaders 'retry-after' 7? null:ifais arrav(Svalue))4$value = $value[0] ?? null;if (ic numeric(<valne)) 1Smessade = strtolover(Se-s>aetMessage.):if (str contains(Smessage, 'daily')) {1t str contains Smessage, "ten secondlv')) <if (str contains(Smessade, "secondly')) ^$this->loq->warning(*(Hubspot] No retry-after header or known message, using default'.?3 Review rate limit handli..Review rate limit handling for HubSpot jo.Here are code reveiw suggestions: Issues"retrvAtter' → 30Now run all affected tests to verify everything passes:Bash Run all affected teststests/Unit/Services/Crm/Hubspot/Pagination/HubspotPaginationServiceTest.php \nCommandmunninginbackaroundwitthTD.hhu@az?vk.Ontoutisbeing_waiitten_to./naivate/tmo//alaude-501/_Isers_1mkas-liminnv-ann//cae817bd_3248-4663-h336-2fi1dBash Check test outoutcat /private/tmp/claude-501/-Users-lukas-jiminny-app/cae817bd-3c48-4c63-b336-2f1d874f2e4d/tasks/bhu9az8vk.output 2>/dev/null | tail -30PHPUnit 11.5.55 by Sebastian Bergmann and contributors.PHP 8.3.30Thouaht for 2c>• All 120 tests pass. Here's a summary of every change made to address the review feedback:Issue 1 - Batch APl calls bypass rate-limit handlingClient.php:305-321 — batchRead0bjects() now wrapsthe $batchConfig['api']->read(...) call inside executeRequest() , so 429s from batch endpoints are caught by thesame circuit-breaker logic. A dedicated catch (RateLimitException $e) { throw $e; } before the general catch ensures the rate-limit exception propagates to the jobmiddleware instead of being swallowed by handleBatchError() andre-wrapped as CrmException.Iccue 2 — Micleadina cache kev nameClient.php:120 - Renamed hubspot:ratelimit:portal:%d → hubspot:ratelimit:confiq:%d. The key was already scoped to the config record ID (not a HubSpot portal ID):the name now matches the intent. The correspondina strina in ClientTest, oho was vodated to match.Issue 3 - isHubspotRateLimit() and parseRetryAfter() visibilityClient.oho:123.137- Both.methods chanqed.from_nwblic tonoivate.Neither is.in the.interface and.nothing.outsideexecuteReuestcalls.them.The direct tests.inClientTest-php were updated to invoke them via ReflectionMethod through two small private helper methods ( callIsHubspotRateLimit, callParseRetryAfter ),preserving all the edge-case coverageIssue 5 - Test constants duolicated from oroduction codewith comments e.o. 'expectedmax"=> 605. // 600 + 5 fitter). If the middleware constants change, these tests will now fail rather than silently pass against stale expectedvaluesIssue 6 - Double-clamoing in RateLimitExcentionPatellimitSycention nho- The dlamoina movedito the constructor (SthisosretrvAfter -mayill SretrayAfterThe oronertvlis no llonder a nromoted narameter (whichssue 7 - Non-obvious RateLimitExcention pathin failediolMatchActivitvCrmData.ohp:172 — Added a three-line comment explaining that RateLimitException onlv reaches failed() when retrvüntil() expires while the iob is stillheld in the released state, because the middleware calls release() rather than re-throwing.9 Ece to focuc or unfocuc ClaudelM Client.ohdth Ask before edits0 Not Committed Yet Ln 120, Col 1 Spaces: 4 UTF-8 LF PHP 8 SignIn 8.3 P 0...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20331
|
|
20332
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20332
|
|
20333
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny...
|
Code
|
Client.php — app — 9 problems in this file • Modif Client.php — app — 9 problems in this file • Modified...
|
NULL
|
20333
|
|
20334
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20334
|
|
20335
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20335
|
|
20336
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20336
|
|
20337
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Lukas Kovalik (2 hours ago)
Lukas Kovalik (2 hours ago)
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20337
|
|
20338
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
Client.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 120, Col 1
git-commit Lukas Kovalik (2 hours ago)
Lukas Kovalik (2 hours ago)
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20338
|
|
20339
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20339
|
|
20340
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik...
|
Code
|
Client.php (Working Tree) (Client.php) — app — 9 p Client.php (Working Tree) (Client.php) — app — 9 problems in this file • Modified...
|
NULL
|
20340
|
|
20341
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G)
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
RateLimitException.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
…
<?php
declare(strict_types=1);
namespace Jiminny\Exceptions;
use Throwable;
class RateLimitException extends RuntimeException
{
private readonly int $retryAfter;
public function __construct(
string $message = '',
int $retryAfter = 1,
?Throwable $previous = null,
) {
parent::__construct($message, 0, $previous);
$this->retryAfter = max(1, $retryAfter);
}
public function getRetryAfter(): int
{
return $this->retryAfter;
}
}
<?php
declare(strict_types=1);
namespace Jiminny\Exceptions;
use Throwable;
class RateLimitException extends RuntimeException
{
private readonly int $retryAfter;
public function __construct(
string $message = '',
int $retryAfter = 1,
?Throwable $previous = null,
) {
parent::__construct($message, 0, $previous);
$this->retryAfter = max(1, $retryAfter);
}
public function getRetryAfter(): int
{
return $this->retryAfter;
}
}
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
Editor Language Status: $(copilot) No inline suggestion available, Inline suggestions
LF
UTF-8
Spaces: 4
Ln 1, Col 1
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob...
|
Code
|
RateLimitException.php — app — Modified
|
NULL
|
20341
|
|
20342
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section...
|
Code
|
RateLimitException.php — app — Modified
|
NULL
|
20342
|
|
20343
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
...
|
Code
|
RateLimitException.php — app — Modified
|
NULL
|
20343
|
|
20344
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U
CreateMockAskJiminnyReportResultCommand.php
app/Console/Commands/Reports
U
RateLimitException.php
app/Exceptions
M
MatchActivityCrmData.php
app/Jobs/Crm
M
Client.php
app/Services/Crm/Hubspot
9, M
logging.php
config
M
favicon.ico
public
U
HandleHubspotRateLimitTest.php
tests/Unit/Jobs/Middleware
M
CanAccessAiReportsTest.php
tests/Unit/Policies
U
ClientTest.php
tests/Unit/Services/Crm/Hubspot
M
Graph Section
GRAPH
GRAPH
JY-20725 add more tests
Lukas Kovalik
JY-20725-handle-HS-search-rate-limit
JY-20725 add tests
Lukas Kovalik
JY-20725 add HS rate limit handling on activities rematching
Lukas Kovalik
Merge pull request #12056 from jiminny/JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
origin/master
Merge branch 'master' into JY-20818-move-AJ-reports-to-separated-datadog-metric
Lukas Kovalik
Merge pull request #12057 from jiminny/JY-20819-increase-download-transctip-rate-limit
mihailmihaylovjiminny
JY-20819: Increase download transcription rate limit
mihail.mihaylov
JY-20818 move ask jiminny reports to its own datadog metric
Lukas Kovalik
Merge pull request #12052 from jiminny/JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge branch 'master' into JY-20817-fix-deleting-old-tracks
mihailmihaylovjiminny
Merge pull request #12047 from jiminny/JY-20662-remove-word-boost
ilian-jiminny
JY-20662 return void
ilian
JY-20662 Removed word_boost parameter from Assembly transcription request
ilian
JY-20817: Fix deleting old tracks
mihail.mihaylov
Merge pull request #12043 from jiminny/JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge branch 'master' into JY-19938-bot-status-in-csv-es
ilian-jiminny
Merge pull request #12044 from jiminny/JY-19938-ES-mapping-update
ilian-jiminny
JY-19938 Added consent information and recording error reason to Elastic Search mapping
ilian
JY-19938 reorder array map
ilian
JY-19938: Address PR review comments
ilian
JY-19938 lint fixes
ilian
HandleHubspotRateLimitTest.php (Working Tree) (HandleHubspotRateLimitTest.php), Editor Group 1
ClientTest.php (Working Tree) (ClientTest.php), Editor Group 1
Client.php (Working Tree) (Client.php), Editor Group 1
RateLimitException.php, preview, Editor Group 1
MatchActivityCrmData.php (Working Tree) (MatchActivityCrmData.php), Editor Group 1
RateLimitException.php (Working Tree) (RateLimitException.php), Editor Group 1
Review rate limit handli…, Editor Group 2
remote
app (Git) - JY-20725-handle-HS-search-rate-limit*, Checkout Branch/Tag...
JY-20725-handle-HS-search-rate-limit*
app (Git) - Synchronize Changes
Errors: 6, Warnings: 11, Infos: 28
6
11
28
Notifications
key, PHP extension: Premium features not active.
8.3, Language Level: 8.3
Sign In
Sign In
PHP
LF
UTF-8
Spaces: 4
Ln 11, Col 1
git-commit Not Committed Yet
Not Committed Yet
Diff editor
Clear
Review rate limit handling for HubSpot job queue
Session history
New session
Message actions
RateLimitException.php#9-9
RateLimitException.php#9-9
Reveiw the diff. Mostly
@app/Jobs/Crm/MatchActivityCrmData.php
,
@app/Exceptions/RateLimitException.php
,
@app/Jobs/Middleware/HandleHubspotRateLimit.php
,
@app/Services/Crm/Hubspot/Client.php
and
@app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php
. The issue is I am trying to make sure that instant spike of MatchActivityCrmData jobs willl handle 429 exceptions from hubspot. Revewi ti carefully and try to see potential issues and bottlenecks. Also RateLimitException was changerd from LogicException to RuntimeException. Is it ok. This exception is used by other paerts of code. Will ti break anything?
Thought for 5s
Thought for 5s
I'll review the implementation carefully. Let me first check how
RateLimitException
is used elsewhere and look at the test files and
PaginationState
.
Bash
Find all usages of RateLimitException outside the changed files
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/app --include="*.php" | grep -v "HandleHubspotRateLimit\|MatchActivityCrmData\|Client.php\|RateLimitException.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:17:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php:68: RateLimitException |
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:15:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/TranscriptionSummary/Jobs/SummarizeTranscriptionJob.php:58: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:19:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/AiActivityType/Jobs/GenerateAiActivityTypeJob.php:76: RateLimitException |
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/HubspotClientInterface.php:50: * @throws RateLimitException When rate limit is hit
/Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Pagination/HubspotPaginationService.php:195: // RateLimitException and other exceptions are re-thrown as-is
Read
PaginationState.php
PaginationState.php
Read
HandleHubspotRateLimitTest.php
HandleHubspotRateLimitTest.php
Bash
Check how RateLimitException is used in GenerateActionItemsJob
IN
grep -rn "RateLimitException\|LogicException\|RuntimeException" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -20
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:13:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php:51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
Bash
Find all catch blocks or instanceof checks for RateLimitException
IN
grep -rn "catch.*RateLimitException\|instanceof RateLimitException" /Users/lukas/jiminny/app/app --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:11: * Job middleware that catches RateLimitException from HubSpot API calls
/Users/lukas/jiminny/app/app/Jobs/Middleware/HandleHubspotRateLimit.php:24: } catch (RateLimitException $e) {
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:149: if (! $e instanceof RateLimitException) {
Bash
Find all test references to RateLimitException
IN
grep -rn "RateLimitException" /Users/lukas/jiminny/app/tests --include="*.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:8:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:13:#[CoversClass(RateLimitException::class)]
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:14:class RateLimitExceptionTest extends TestCase
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:18: $exception = new RateLimitException();
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:28: $exception = new RateLimitException('Hubspot rate limit', 30, $previous);
/Users/lukas/jiminny/app/tests/Unit/Exceptions/RateLimitExceptionTest.php:52: $exception = new RateLimitException('test', $input);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:10:use Jiminny\Exceptions\RateLimitException;
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:49: public function testPropagatesNonRateLimitExceptions(): void
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:103: throw new RateLimitException('rate limited', $retryAfter);
/Users/lukas/jiminny/app/tests/Unit/Jobs/Middleware/HandleHubspotRateLimitTest.php:150: throw new RateLimitException('rate limited', 1);
Bash
Check retry/failure handling in GenerateActionItemsJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/ActionItems/Jobs/GenerateActionItemsJob.php | head -30
Copy code to clipboard
OUT
13:use Jiminny\Exceptions\RateLimitException;
51: GuzzleException|HttpBadRequestException|JsonException|RateLimitException|UnexpectedValueException $exception
56: $this->release(30);
Bash
Check retry/failure handling in GenerateAiCallScoringJob
IN
grep -n "RateLimitException\|retryUntil\|backoff\|maxExceptions\|maxTries\|release\|failed" /Users/lukas/jiminny/app/app/Component/AiCallScoring/Jobs/GenerateAiCallScoringJob.php | head -30
Copy code to clipboard
OUT
17:use Jiminny\Exceptions\RateLimitException;
68: RateLimitException |
75: $this->release(30);
Bash
Check retry config in MatchActivityCrmData
IN
grep -n "retryAfter\|RETRY_WINDOW\|retryUntil\|maxTries" /Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php
Copy code to clipboard
OUT
34: private const int RETRY_WINDOW_MINUTES = 30;
45: public function retryUntil(): \DateTimeInterface
47: return now()->addMinutes(self::RETRY_WINDOW_MINUTES);
77: return self::RETRY_WINDOW_MINUTES * 60 + 60;
Bash
Check the cache key format
IN
grep -n "getRateLimitCacheKey\|hubspot:ratelimit" /Users/lukas/jiminny/app/app/Services/Crm/Hubspot/Client.php
Copy code to clipboard
OUT
78: $cacheKey = $this->getRateLimitCacheKey();
110: private function getRateLimitCacheKey(): string
112: return sprintf('hubspot:ratelimit:portal:%d', $this->config->getId());
Bash
Find other jobs that might use the rate limit middleware
IN
grep -rn "class.*Middleware\|HandleHubspotRateLimit" /Users/lukas/jiminny/app/app/Jobs --include="*.php" | grep -v "HandleHubspotRateLimit.php"
Copy code to clipboard
OUT
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:18:use Jiminny\Jobs\Middleware\HandleHubspotRateLimit;
/Users/lukas/jiminny/app/app/Jobs/Crm/MatchActivityCrmData.php:42: return [new HandleHubspotRateLimit()];
Read
HubspotClientInterface.php
HubspotClientInterface.php
Thought for 73s
Thought for 73s
Now I have a complete picture. Here's my full analysis:
Review Summary
Review Summary
RuntimeException vs LogicException — Safe ✓
RuntimeException
vs
LogicException
— Safe ✓
The change is semantically correct and won't break anything.
LogicException
is for programmer errors (invalid arguments, contract violations). A rate limit from an external API is a runtime condition —
RuntimeException
is the right parent. All callers catch
RateLimitException
by name explicitly (union catches in
GenerateActionItemsJob
,
GenerateAiCallScoringJob
,
SummarizeTranscriptionJob
,
GenerateAiActivityTypeJob
) or check
instanceof RateLimitException
— none relies on the parent class type, so nothing breaks.
Issue 1 — Circuit breaker stores stale delay (most important bug)
Issue 1 — Circuit breaker stores stale delay (most important bug)
Client.php:94
Client.php:94
Copy code to clipboard
Redis::setex($cacheKey, $retryAfter, (string) $retryAfter);
The TTL is
$retryAfter
seconds, but the
stored value
is also the original
$retryAfter
. If a second job reads this cache 20 seconds later, it throws
RateLimitException(retryAfter=30)
and gets delayed 30 more seconds — even though only 10 remain. In a spike of 100 jobs, each subsequent job inflates its wait.
Fix:
store the absolute expiry timestamp, reconstruct the remaining delay when reading:
Copy code to clipboard
// on 429:
Redis::setex($cacheKey, $retryAfter, (string) (time() + $retryAfter));
// in the cached-read branch:
$remaining = max(1, (int) $cachedRetryAfter - time());
throw new RateLimitException('Hubspot rate limit (cached circuit-breaker)', $remaining);
Issue 2 — retryUntil() window may expire under sustained rate limiting
Issue 2 —
retryUntil()
window may expire under sustained rate limiting
MatchActivityCrmData.php:45-47
MatchActivityCrmData.php:45-47
retryUntil()
is calculated as
now() + 30 min
at
dispatch time
and stored in the job payload. With
MAX_RETRY_DELAY = 600s
, you only get ~3 rate-limit-triggered releases before the 30-minute window closes and the job is permanently failed. For a heavy 429 spike (daily limit, parsed as 600s), this can easily happen.
Consider increasing
RETRY_WINDOW_MINUTES
to 60 or more, or making it configurable. The
uniqueFor()
value should be updated to match.
Issue 3 — Only search() uses the circuit breaker, but other methods can still hit 429
Issue 3 — Only
search()
uses the circuit breaker, but other methods can still hit 429
Client.php:76-108
Client.php:76-108
executeRequest()
is only called from
search()
. Methods like
getOpportunityById()
,
batchReadObjects()
,
getContactByEmail()
bypass it entirely. If HubSpot's 429 is account-wide (not search-endpoint-specific), those methods will keep firing even while the circuit breaker is supposedly open. This is likely fine because the search endpoint has its own tighter quota (5 RPS), but it's worth being explicit in a comment.
Issue 4 — object $job type in middleware has no compile-time safety
Issue 4 —
object $job
type in middleware has no compile-time safety
HandleHubspotRateLimit.php:20...
|
Code
|
RateLimitException.php (Working Tree) (RateLimitEx RateLimitException.php (Working Tree) (RateLimitException.php) — app — Modified...
|
NULL
|
20344
|
|
20357
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
...
|
Code
|
Review rate limit handli… — app
|
NULL
|
20357
|
|
20358
|
Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧ Explorer (⇧⌘E)
Search (⇧⌘F)
Source Control (⌃⇧G) - 17 pending changes
17
Run and Debug (⇧⌘D)
Remote Explorer
Extensions (⇧⌘X) - 2 require update
2
Testing
Claude Code
SOURCE CONTROL
SOURCE CONTROL
Changes Section
CHANGES
CHANGES
Commit Changes on "JY-20725-handle-HS-search-rate-limit"
Commit
More Actions...
Changes
17
.env.local
M
.env.nikilocal
U
.env.other
U
ids.txt
U
raw_sql_query.sql
U
WEBHOOK_FILTERING_IMPLEMENTATION.md
U
JiminnyDebugCommand.php
app/Console/Commands
M
SimulateWebhooksCommand.php
app/Console/Commands/Crm/Hubspot
U...
|
Code
|
Review rate limit handli… — app
|
NULL
|
20358
|