|
50361
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY 20893 chunk control per update target by Vasil-</tabTitle>” with “<selection>@@ -20,6 +20,9 @@ class TargetEntitiesSelectorTest extends TestCase202021public function testTargetStaleChunksFirst(): void21public function testTargetStaleChunksFirst(): void22 {22 {23+$mainChunkSize = 200;24+$staleChunkSize = 50;25+23$batchStatusManager = $this->createMock(BatchStatusInterface::class);26$batchStatusManager = $this->createMock(BatchStatusInterface::class);242725$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];28$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];@@ -28,13 +31,13 @@ public function testTargetStaleChunksFirst(): void28'scheduled-batch-2' => false,31'scheduled-batch-2' => false,29 ];32 ];30// Exactly one batch33// Exactly one batch31-$scheduledEntityIds = range(101, TargetEntitiesSelector::ES_CHUNK_SIZE + 100);34+$scheduledEntityIds = range(101, $mainChunkSize + 100);323533-$batchStatusManager->expects($this->atLeastOnce())36+$batchStatusManager->expects($this->exactly(5))34 ->method('getAllScheduledBatches')37 ->method('getAllScheduledBatches')35 ->willReturnCallback(fn () => $this->scheduledBatches);38 ->willReturnCallback(fn () => $this->scheduledBatches);363937-$batchStatusManager->expects($this->atLeast(2))40+$batchStatusManager->expects($this->exactly(14))38 ->method('isBatchStillInProgress')41 ->method('isBatchStillInProgress')39 ->willReturnCallback(function ($batchName) {42 ->willReturnCallback(function ($batchName) {40return $this->batchesInProgress[$batchName];43return $this->batchesInProgress[$batchName];@@ -45,13 +48,13 @@ public function testTargetStaleChunksFirst(): void45 * I.e. 100 items in chunks of 10 will produce 10 iterations,48 * I.e. 100 items in chunks of 10 will produce 10 iterations,46 * full batch of 250 ill be processed in 5 chunks of 50 items49 * full batch of 250 ill be processed in 5 chunks of 50 items47 */50 */48-$z = (int) floor(TargetEntitiesSelector::ES_CHUNK_SIZE / TargetEntitiesSelector::SMALL_CHUNK_SIZE);51+$z = (int) floor($mainChunkSize / $staleChunkSize);495250-$batchStatusManager->expects($this->atLeast($z))53+$batchStatusManager->expects($this->exactly($z + 1))51 ->method('extractEntityIdsFromBatch')54 ->method('extractEntityIdsFromBatch')52- ->with('scheduled-batch-2', TargetEntitiesSelector::SMALL_CHUNK_SIZE)55+ ->with('scheduled-batch-2', $staleChunkSize)53- ->willReturnCallback(function () use (&$scheduledEntityIds) {56+ ->willReturnCallback(function () use (&$scheduledEntityIds, $staleChunkSize) {54-return array_splice($scheduledEntityIds, 0, TargetEntitiesSelector::SMALL_CHUNK_SIZE);57+return array_splice($scheduledEntityIds, 0, $staleChunkSize);55 });58 });565957 Redis::shouldReceive('get')60 Redis::shouldReceive('get')@@ -63,15 +66,15 @@ public function testTargetStaleChunksFirst(): void63$highPriorityEnd = 5030;66$highPriorityEnd = 5030;64$highPriorityRange = range($highPriorityStart, $highPriorityEnd);67$highPriorityRange = range($highPriorityStart, $highPriorityEnd);656866-$normalPrioritySize = TargetEntitiesSelector::ES_CHUNK_SIZE - count($highPriorityRange);69+$normalPrioritySize = $mainChunkSize - count($highPriorityRange);677068$normalPriorityStart = 1;71$normalPriorityStart = 1;69$normalPriorityEnd = 50;72$normalPriorityEnd = 50;70$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);73$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);717472 Redis::shouldReceive('spop')75 Redis::shouldReceive('spop')73 ->once()76 ->once()74- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)77+ ->with('activities-for-update-priority', $mainChunkSize)75 ->andReturn($highPriorityRange);78 ->andReturn($highPriorityRange);767977 Redis::shouldReceive('spop')80 Redis::shouldReceive('spop')@@ -87,8 +90,8 @@ public function testTargetStaleChunksFirst(): void87$normalPriority = $selectionList->getNormalPriorityEntityId();90$normalPriority = $selectionList->getNormalPriorityEntityId();88$highPriority = $selectionList->getHighPriorityEntityIds();91$highPriority = $selectionList->getHighPriorityEntityIds();899290-$startOffsetId = 101 + (($i - 1) * TargetEntitiesSelector::SMALL_CHUNK_SIZE);93+$startOffsetId = 101 + (($i - 1) * $staleChunkSize);91-$endOffsetId = 100 + ($i * TargetEntitiesSelector::SMALL_CHUNK_SIZE);94+$endOffsetId = 100 + ($i * $staleChunkSize);929593if ($i < ($z + 1)) {96if ($i < ($z + 1)) {94/**97/**@@ -126,8 +129,10 @@ public function testTargetStaleChunksFirst(): void126129127public function testOnlyHighPriorityEntitiesToProcess(): void130public function testOnlyHighPriorityEntitiesToProcess(): void128 {131 {132+$mainChunkSize = 200;133+129$batchStatusManager = $this->createMock(BatchStatusInterface::class);134$batchStatusManager = $this->createMock(BatchStatusInterface::class);130-$batchStatusManager->expects($this->atLeastOnce())135+$batchStatusManager->expects($this->once())131 ->method('getAllScheduledBatches')136 ->method('getAllScheduledBatches')132 ->willReturn([]);137 ->willReturn([]);133138@@ -137,12 +142,12 @@ public function testOnlyHighPriorityEntitiesToProcess(): void137142138 Redis::shouldReceive('spop')143 Redis::shouldReceive('spop')139 ->never()144 ->never()140- ->with('activities-for-update', TargetEntitiesSelector::ES_CHUNK_SIZE);145+ ->with('activities-for-update', $mainChunkSize);141146142 Redis::shouldReceive('spop')147 Redis::shouldReceive('spop')143 ->once()148 ->once()144- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)149+ ->with('activities-for-update-priority', $mainChunkSize)145- ->andReturn(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE));150+ ->andReturn(range(1, $mainChunkSize));146151147$selector = $this->getSelector($batchStatusManager);152$selector = $this->getSelector($batchStatusManager);148153@@ -153,7 +158,7 @@ public function testOnlyHighPriorityEntitiesToProcess(): void153$this->assertEmpty($normalPriority);158$this->assertEmpty($normalPriority);154$this->assertNotEmpty($highPriority);159$this->assertNotEmpty($highPriority);155160156-$this->assertEquals(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE), $highPriority);161+$this->assertEquals(range(1, $mainChunkSize), $highPriority);157 }162 }158163159private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector164private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY 20893 chunk control per update target by Vasil-</tabTitle>” with “<selection>@@ -20,6 +20,9 @@ class TargetEntitiesSelectorTest extends TestCase202021public function testTargetStaleChunksFirst(): void21public function testTargetStaleChunksFirst(): void22 {22 {23+$mainChunkSize = 200;24+$staleChunkSize = 50;25+23$batchStatusManager = $this->createMock(BatchStatusInterface::class);26$batchStatusManager = $this->createMock(BatchStatusInterface::class);242725$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];28$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];@@ -28,13 +31,13 @@ public function testTargetStaleChunksFirst(): void28'scheduled-batch-2' => false,31'scheduled-batch-2' => false,29 ];32 ];30// Exactly one batch33// Exactly one batch31-$scheduledEntityIds = range(101, TargetEntitiesSelector::ES_CHUNK_SIZE + 100);34+$scheduledEntityIds = range(101, $mainChunkSize + 100);323533-$batchStatusManager->expects($this->atLeastOnce())36+$batchStatusManager->expects($this->exactly(5))34 ->method('getAllScheduledBatches')37 ->method('getAllScheduledBatches')35 ->willReturnCallback(fn () => $this->scheduledBatches);38 ->willReturnCallback(fn () => $this->scheduledBatches);363937-$batchStatusManager->expects($this->atLeast(2))40+$batchStatusManager->expects($this->exactly(14))38 ->method('isBatchStillInProgress')41 ->method('isBatchStillInProgress')39 ->willReturnCallback(function ($batchName) {42 ->willReturnCallback(function ($batchName) {40return $this->batchesInProgress[$batchName];43return $this->batchesInProgress[$batchName];@@ -45,13 +48,13 @@ public function testTargetStaleChunksFirst(): void45 * I.e. 100 items in chunks of 10 will produce 10 iterations,48 * I.e. 100 items in chunks of 10 will produce 10 iterations,46 * full batch of 250 ill be processed in 5 chunks of 50 items49 * full batch of 250 ill be processed in 5 chunks of 50 items47 */50 */48-$z = (int) floor(TargetEntitiesSelector::ES_CHUNK_SIZE / TargetEntitiesSelector::SMALL_CHUNK_SIZE);51+$z = (int) floor($mainChunkSize / $staleChunkSize);495250-$batchStatusManager->expects($this->atLeast($z))53+$batchStatusManager->expects($this->exactly($z + 1))51 ->method('extractEntityIdsFromBatch')54 ->method('extractEntityIdsFromBatch')52- ->with('scheduled-batch-2', TargetEntitiesSelector::SMALL_CHUNK_SIZE)55+ ->with('scheduled-batch-2', $staleChunkSize)53- ->willReturnCallback(function () use (&$scheduledEntityIds) {56+ ->willReturnCallback(function () use (&$scheduledEntityIds, $staleChunkSize) {54-return array_splice($scheduledEntityIds, 0, TargetEntitiesSelector::SMALL_CHUNK_SIZE);57+return array_splice($scheduledEntityIds, 0, $staleChunkSize);55 });58 });565957 Redis::shouldReceive('get')60 Redis::shouldReceive('get')@@ -63,15 +66,15 @@ public function testTargetStaleChunksFirst(): void63$highPriorityEnd = 5030;66$highPriorityEnd = 5030;64$highPriorityRange = range($highPriorityStart, $highPriorityEnd);67$highPriorityRange = range($highPriorityStart, $highPriorityEnd);656866-$normalPrioritySize = TargetEntitiesSelector::ES_CHUNK_SIZE - count($highPriorityRange);69+$normalPrioritySize = $mainChunkSize - count($highPriorityRange);677068$normalPriorityStart = 1;71$normalPriorityStart = 1;69$normalPriorityEnd = 50;72$normalPriorityEnd = 50;70$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);73$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);717472 Redis::shouldReceive('spop')75 Redis::shouldReceive('spop')73 ->once()76 ->once()74- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)77+ ->with('activities-for-update-priority', $mainChunkSize)75 ->andReturn($highPriorityRange);78 ->andReturn($highPriorityRange);767977 Redis::shouldReceive('spop')80 Redis::shouldReceive('spop')@@ -87,8 +90,8 @@ public function testTargetStaleChunksFirst(): void87$normalPriority = $selectionList->getNormalPriorityEntityId();90$normalPriority = $selectionList->getNormalPriorityEntityId();88$highPriority = $selectionList->getHighPriorityEntityIds();91$highPriority = $selectionList->getHighPriorityEntityIds();899290-$startOffsetId = 101 + (($i - 1) * TargetEntitiesSelector::SMALL_CHUNK_SIZE);93+$startOffsetId = 101 + (($i - 1) * $staleChunkSize);91-$endOffsetId = 100 + ($i * TargetEntitiesSelector::SMALL_CHUNK_SIZE);94+$endOffsetId = 100 + ($i * $staleChunkSize);929593if ($i < ($z + 1)) {96if ($i < ($z + 1)) {94/**97/**@@ -126,8 +129,10 @@ public function testTargetStaleChunksFirst(): void126129127public function testOnlyHighPriorityEntitiesToProcess(): void130public function testOnlyHighPriorityEntitiesToProcess(): void128 {131 {132+$mainChunkSize = 200;133+129$batchStatusManager = $this->createMock(BatchStatusInterface::class);134$batchStatusManager = $this->createMock(BatchStatusInterface::class);130-$batchStatusManager->expects($this->atLeastOnce())135+$batchStatusManager->expects($this->once())131 ->method('getAllScheduledBatches')136 ->method('getAllScheduledBatches')132 ->willReturn([]);137 ->willReturn([]);133138@@ -137,12 +142,12 @@ public function testOnlyHighPriorityEntitiesToProcess(): void137142138 Redis::shouldReceive('spop')143 Redis::shouldReceive('spop')139 ->never()144 ->never()140- ->with('activities-for-update', TargetEntitiesSelector::ES_CHUNK_SIZE);145+ ->with('activities-for-update', $mainChunkSize);141146142 Redis::shouldReceive('spop')147 Redis::shouldReceive('spop')143 ->once()148 ->once()144- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)149+ ->with('activities-for-update-priority', $mainChunkSize)145- ->andReturn(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE));150+ ->andReturn(range(1, $mainChunkSize));146151147$selector = $this->getSelector($batchStatusManager);152$selector = $this->getSelector($batchStatusManager);148153@@ -153,7 +158,7 @@ public function testOnlyHighPriorityEntitiesToProcess(): void153$this->assertEmpty($normalPriority);158$this->assertEmpty($normalPriority);154$this->assertNotEmpty($highPriority);159$this->assertNotEmpty($highPriority);155160156-$this->assertEquals(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE), $highPriority);161+$this->assertEquals(range(1, $mainChunkSize), $highPriority);157 }162 }158163159private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector164private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Summary of Changes
Summary of Changes
The selected diff updates
TargetEntitiesSelectorTest
by replacing hardcoded class constants with local variables (
$mainChunkSize
and
$staleChunkSize
) to make the tests more flexible, while explicitly tightening the expected call counts on mock objects.
Detailed Modifications
Detailed Modifications
1. Test Parameterization
1. Test Parameterization
Introduced Local Variables:
Replaced references to
TargetEntitiesSelector::ES_CHUNK_SIZE
with
$mainChunkSize = 200
and
TargetEntitiesSelector::SMALL_CHUNK_SIZE
with
$staleChunkSize = 50
.
Dynamic Range and Offset Calculations:
Updated array ranges, Redis
spop
arguments, and iteration offsets to derive dynamically from these local variables instead of hardcoded class constants.
2. Mock Expectation Hardening
2. Mock Expectation Hardening
getAllScheduledBatches
:
Changed expectations from
atLeastOnce()
to explicit strict counts:
exactly(5)
in
testTargetStaleChunksFirst
once()
in
testOnlyHighPriorityEntitiesToProcess
isBatchStillInProgress
:
Changed expectation from
atLeast(2)
to
exactly(14)
.
extractEntityIdsFromBatch
:
Changed expectation from
atLeast($z)
to
exactly($z + 1)
.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said review PR, non test files and try to find some potential issues and bottlenecks
You said
review PR, non test files and try to find some potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I’d be happy to review the main source files for potential issues and performance bottlenecks! However, the snippet you shared previously only contained the
TargetEntitiesSelectorTest
file. I don't currently have access to the rest of the Pull Request.
Could you please paste the diff or the code for the non-test files (such as
TargetEntitiesSelector.php
) so I can take a look?
Good response
Bad response
Share & export
Copy
Show more options
Show the uploaded image in a lightbox
Copy prompt
Edit
You said Make code review
You said...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086/changes
|
50361
|
|
50391
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086/changes
|
50391
|
|
50392
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086/changes
|
50392
|
|
50393
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY 20893 chunk control per update target by Vasil-</tabTitle>” with “<selection>@@ -20,6 +20,9 @@ class TargetEntitiesSelectorTest extends TestCase202021public function testTargetStaleChunksFirst(): void21public function testTargetStaleChunksFirst(): void22 {22 {23+$mainChunkSize = 200;24+$staleChunkSize = 50;25+23$batchStatusManager = $this->createMock(BatchStatusInterface::class);26$batchStatusManager = $this->createMock(BatchStatusInterface::class);242725$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];28$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];@@ -28,13 +31,13 @@ public function testTargetStaleChunksFirst(): void28'scheduled-batch-2' => false,31'scheduled-batch-2' => false,29 ];32 ];30// Exactly one batch33// Exactly one batch31-$scheduledEntityIds = range(101, TargetEntitiesSelector::ES_CHUNK_SIZE + 100);34+$scheduledEntityIds = range(101, $mainChunkSize + 100);323533-$batchStatusManager->expects($this->atLeastOnce())36+$batchStatusManager->expects($this->exactly(5))34 ->method('getAllScheduledBatches')37 ->method('getAllScheduledBatches')35 ->willReturnCallback(fn () => $this->scheduledBatches);38 ->willReturnCallback(fn () => $this->scheduledBatches);363937-$batchStatusManager->expects($this->atLeast(2))40+$batchStatusManager->expects($this->exactly(14))38 ->method('isBatchStillInProgress')41 ->method('isBatchStillInProgress')39 ->willReturnCallback(function ($batchName) {42 ->willReturnCallback(function ($batchName) {40return $this->batchesInProgress[$batchName];43return $this->batchesInProgress[$batchName];@@ -45,13 +48,13 @@ public function testTargetStaleChunksFirst(): void45 * I.e. 100 items in chunks of 10 will produce 10 iterations,48 * I.e. 100 items in chunks of 10 will produce 10 iterations,46 * full batch of 250 ill be processed in 5 chunks of 50 items49 * full batch of 250 ill be processed in 5 chunks of 50 items47 */50 */48-$z = (int) floor(TargetEntitiesSelector::ES_CHUNK_SIZE / TargetEntitiesSelector::SMALL_CHUNK_SIZE);51+$z = (int) floor($mainChunkSize / $staleChunkSize);495250-$batchStatusManager->expects($this->atLeast($z))53+$batchStatusManager->expects($this->exactly($z + 1))51 ->method('extractEntityIdsFromBatch')54 ->method('extractEntityIdsFromBatch')52- ->with('scheduled-batch-2', TargetEntitiesSelector::SMALL_CHUNK_SIZE)55+ ->with('scheduled-batch-2', $staleChunkSize)53- ->willReturnCallback(function () use (&$scheduledEntityIds) {56+ ->willReturnCallback(function () use (&$scheduledEntityIds, $staleChunkSize) {54-return array_splice($scheduledEntityIds, 0, TargetEntitiesSelector::SMALL_CHUNK_SIZE);57+return array_splice($scheduledEntityIds, 0, $staleChunkSize);55 });58 });565957 Redis::shouldReceive('get')60 Redis::shouldReceive('get')@@ -63,15 +66,15 @@ public function testTargetStaleChunksFirst(): void63$highPriorityEnd = 5030;66$highPriorityEnd = 5030;64$highPriorityRange = range($highPriorityStart, $highPriorityEnd);67$highPriorityRange = range($highPriorityStart, $highPriorityEnd);656866-$normalPrioritySize = TargetEntitiesSelector::ES_CHUNK_SIZE - count($highPriorityRange);69+$normalPrioritySize = $mainChunkSize - count($highPriorityRange);677068$normalPriorityStart = 1;71$normalPriorityStart = 1;69$normalPriorityEnd = 50;72$normalPriorityEnd = 50;70$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);73$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);717472 Redis::shouldReceive('spop')75 Redis::shouldReceive('spop')73 ->once()76 ->once()74- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)77+ ->with('activities-for-update-priority', $mainChunkSize)75 ->andReturn($highPriorityRange);78 ->andReturn($highPriorityRange);767977 Redis::shouldReceive('spop')80 Redis::shouldReceive('spop')@@ -87,8 +90,8 @@ public function testTargetStaleChunksFirst(): void87$normalPriority = $selectionList->getNormalPriorityEntityId();90$normalPriority = $selectionList->getNormalPriorityEntityId();88$highPriority = $selectionList->getHighPriorityEntityIds();91$highPriority = $selectionList->getHighPriorityEntityIds();899290-$startOffsetId = 101 + (($i - 1) * TargetEntitiesSelector::SMALL_CHUNK_SIZE);93+$startOffsetId = 101 + (($i - 1) * $staleChunkSize);91-$endOffsetId = 100 + ($i * TargetEntitiesSelector::SMALL_CHUNK_SIZE);94+$endOffsetId = 100 + ($i * $staleChunkSize);929593if ($i < ($z + 1)) {96if ($i < ($z + 1)) {94/**97/**@@ -126,8 +129,10 @@ public function testTargetStaleChunksFirst(): void126129127public function testOnlyHighPriorityEntitiesToProcess(): void130public function testOnlyHighPriorityEntitiesToProcess(): void128 {131 {132+$mainChunkSize = 200;133+129$batchStatusManager = $this->createMock(BatchStatusInterface::class);134$batchStatusManager = $this->createMock(BatchStatusInterface::class);130-$batchStatusManager->expects($this->atLeastOnce())135+$batchStatusManager->expects($this->once())131 ->method('getAllScheduledBatches')136 ->method('getAllScheduledBatches')132 ->willReturn([]);137 ->willReturn([]);133138@@ -137,12 +142,12 @@ public function testOnlyHighPriorityEntitiesToProcess(): void137142138 Redis::shouldReceive('spop')143 Redis::shouldReceive('spop')139 ->never()144 ->never()140- ->with('activities-for-update', TargetEntitiesSelector::ES_CHUNK_SIZE);145+ ->with('activities-for-update', $mainChunkSize);141146142 Redis::shouldReceive('spop')147 Redis::shouldReceive('spop')143 ->once()148 ->once()144- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)149+ ->with('activities-for-update-priority', $mainChunkSize)145- ->andReturn(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE));150+ ->andReturn(range(1, $mainChunkSize));146151147$selector = $this->getSelector($batchStatusManager);152$selector = $this->getSelector($batchStatusManager);148153@@ -153,7 +158,7 @@ public function testOnlyHighPriorityEntitiesToProcess(): void153$this->assertEmpty($normalPriority);158$this->assertEmpty($normalPriority);154$this->assertNotEmpty($highPriority);159$this->assertNotEmpty($highPriority);155160156-$this->assertEquals(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE), $highPriority);161+$this->assertEquals(range(1, $mainChunkSize), $highPriority);157 }162 }158163159private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector164private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY 20893 chunk control per update target by Vasil-</tabTitle>” with “<selection>@@ -20,6 +20,9 @@ class TargetEntitiesSelectorTest extends TestCase202021public function testTargetStaleChunksFirst(): void21public function testTargetStaleChunksFirst(): void22 {22 {23+$mainChunkSize = 200;24+$staleChunkSize = 50;25+23$batchStatusManager = $this->createMock(BatchStatusInterface::class);26$batchStatusManager = $this->createMock(BatchStatusInterface::class);242725$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];28$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];@@ -28,13 +31,13 @@ public function testTargetStaleChunksFirst(): void28'scheduled-batch-2' => false,31'scheduled-batch-2' => false,29 ];32 ];30// Exactly one batch33// Exactly one batch31-$scheduledEntityIds = range(101, TargetEntitiesSelector::ES_CHUNK_SIZE + 100);34+$scheduledEntityIds = range(101, $mainChunkSize + 100);323533-$batchStatusManager->expects($this->atLeastOnce())36+$batchStatusManager->expects($this->exactly(5))34 ->method('getAllScheduledBatches')37 ->method('getAllScheduledBatches')35 ->willReturnCallback(fn () => $this->scheduledBatches);38 ->willReturnCallback(fn () => $this->scheduledBatches);363937-$batchStatusManager->expects($this->atLeast(2))40+$batchStatusManager->expects($this->exactly(14))38 ->method('isBatchStillInProgress')41 ->method('isBatchStillInProgress')39 ->willReturnCallback(function ($batchName) {42 ->willReturnCallback(function ($batchName) {40return $this->batchesInProgress[$batchName];43return $this->batchesInProgress[$batchName];@@ -45,13 +48,13 @@ public function testTargetStaleChunksFirst(): void45 * I.e. 100 items in chunks of 10 will produce 10 iterations,48 * I.e. 100 items in chunks of 10 will produce 10 iterations,46 * full batch of 250 ill be processed in 5 chunks of 50 items49 * full batch of 250 ill be processed in 5 chunks of 50 items47 */50 */48-$z = (int) floor(TargetEntitiesSelector::ES_CHUNK_SIZE / TargetEntitiesSelector::SMALL_CHUNK_SIZE);51+$z = (int) floor($mainChunkSize / $staleChunkSize);495250-$batchStatusManager->expects($this->atLeast($z))53+$batchStatusManager->expects($this->exactly($z + 1))51 ->method('extractEntityIdsFromBatch')54 ->method('extractEntityIdsFromBatch')52- ->with('scheduled-batch-2', TargetEntitiesSelector::SMALL_CHUNK_SIZE)55+ ->with('scheduled-batch-2', $staleChunkSize)53- ->willReturnCallback(function () use (&$scheduledEntityIds) {56+ ->willReturnCallback(function () use (&$scheduledEntityIds, $staleChunkSize) {54-return array_splice($scheduledEntityIds, 0, TargetEntitiesSelector::SMALL_CHUNK_SIZE);57+return array_splice($scheduledEntityIds, 0, $staleChunkSize);55 });58 });565957 Redis::shouldReceive('get')60 Redis::shouldReceive('get')@@ -63,15 +66,15 @@ public function testTargetStaleChunksFirst(): void63$highPriorityEnd = 5030;66$highPriorityEnd = 5030;64$highPriorityRange = range($highPriorityStart, $highPriorityEnd);67$highPriorityRange = range($highPriorityStart, $highPriorityEnd);656866-$normalPrioritySize = TargetEntitiesSelector::ES_CHUNK_SIZE - count($highPriorityRange);69+$normalPrioritySize = $mainChunkSize - count($highPriorityRange);677068$normalPriorityStart = 1;71$normalPriorityStart = 1;69$normalPriorityEnd = 50;72$normalPriorityEnd = 50;70$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);73$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);717472 Redis::shouldReceive('spop')75 Redis::shouldReceive('spop')73 ->once()76 ->once()74- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)77+ ->with('activities-for-update-priority', $mainChunkSize)75 ->andReturn($highPriorityRange);78 ->andReturn($highPriorityRange);767977 Redis::shouldReceive('spop')80 Redis::shouldReceive('spop')@@ -87,8 +90,8 @@ public function testTargetStaleChunksFirst(): void87$normalPriority = $selectionList->getNormalPriorityEntityId();90$normalPriority = $selectionList->getNormalPriorityEntityId();88$highPriority = $selectionList->getHighPriorityEntityIds();91$highPriority = $selectionList->getHighPriorityEntityIds();899290-$startOffsetId = 101 + (($i - 1) * TargetEntitiesSelector::SMALL_CHUNK_SIZE);93+$startOffsetId = 101 + (($i - 1) * $staleChunkSize);91-$endOffsetId = 100 + ($i * TargetEntitiesSelector::SMALL_CHUNK_SIZE);94+$endOffsetId = 100 + ($i * $staleChunkSize);929593if ($i < ($z + 1)) {96if ($i < ($z + 1)) {94/**97/**@@ -126,8 +129,10 @@ public function testTargetStaleChunksFirst(): void126129127public function testOnlyHighPriorityEntitiesToProcess(): void130public function testOnlyHighPriorityEntitiesToProcess(): void128 {131 {132+$mainChunkSize = 200;133+129$batchStatusManager = $this->createMock(BatchStatusInterface::class);134$batchStatusManager = $this->createMock(BatchStatusInterface::class);130-$batchStatusManager->expects($this->atLeastOnce())135+$batchStatusManager->expects($this->once())131 ->method('getAllScheduledBatches')136 ->method('getAllScheduledBatches')132 ->willReturn([]);137 ->willReturn([]);133138@@ -137,12 +142,12 @@ public function testOnlyHighPriorityEntitiesToProcess(): void137142138 Redis::shouldReceive('spop')143 Redis::shouldReceive('spop')139 ->never()144 ->never()140- ->with('activities-for-update', TargetEntitiesSelector::ES_CHUNK_SIZE);145+ ->with('activities-for-update', $mainChunkSize);141146142 Redis::shouldReceive('spop')147 Redis::shouldReceive('spop')143 ->once()148 ->once()144- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)149+ ->with('activities-for-update-priority', $mainChunkSize)145- ->andReturn(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE));150+ ->andReturn(range(1, $mainChunkSize));146151147$selector = $this->getSelector($batchStatusManager);152$selector = $this->getSelector($batchStatusManager);148153@@ -153,7 +158,7 @@ public function testOnlyHighPriorityEntitiesToProcess(): void153$this->assertEmpty($normalPriority);158$this->assertEmpty($normalPriority);154$this->assertNotEmpty($highPriority);159$this->assertNotEmpty($highPriority);155160156-$this->assertEquals(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE), $highPriority);161+$this->assertEquals(range(1, $mainChunkSize), $highPriority);157 }162 }158163159private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector164private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Summary of Changes
Summary of Changes
The selected diff updates
TargetEntitiesSelectorTest
by replacing hardcoded class constants with local variables (
$mainChunkSize
and
$staleChunkSize
) to make the tests more flexible, while explicitly tightening the expected call counts on mock objects.
Detailed Modifications
Detailed Modifications
1. Test Parameterization
1. Test Parameterization
Introduced Local Variables:
Replaced references to
TargetEntitiesSelector::ES_CHUNK_SIZE
with
$mainChunkSize = 200
and
TargetEntitiesSelector::SMALL_CHUNK_SIZE
with
$staleChunkSize = 50
.
Dynamic Range and Offset Calculations:
Updated array ranges, Redis
spop
arguments, and iteration offsets to derive dynamically from these local variables instead of hardcoded class constants.
2. Mock Expectation Hardening
2. Mock Expectation Hardening
getAllScheduledBatches
:
Changed expectations from
atLeastOnce()
to explicit strict counts:
exactly(5)
in...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086/changes
|
50393
|
|
50394
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY 20893 chunk control per update target by Vasil-</tabTitle>” with “<selection>@@ -20,6 +20,9 @@ class TargetEntitiesSelectorTest extends TestCase202021public function testTargetStaleChunksFirst(): void21public function testTargetStaleChunksFirst(): void22 {22 {23+$mainChunkSize = 200;24+$staleChunkSize = 50;25+23$batchStatusManager = $this->createMock(BatchStatusInterface::class);26$batchStatusManager = $this->createMock(BatchStatusInterface::class);242725$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];28$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];@@ -28,13 +31,13 @@ public function testTargetStaleChunksFirst(): void28'scheduled-batch-2' => false,31'scheduled-batch-2' => false,29 ];32 ];30// Exactly one batch33// Exactly one batch31-$scheduledEntityIds = range(101, TargetEntitiesSelector::ES_CHUNK_SIZE + 100);34+$scheduledEntityIds = range(101, $mainChunkSize + 100);323533-$batchStatusManager->expects($this->atLeastOnce())36+$batchStatusManager->expects($this->exactly(5))34 ->method('getAllScheduledBatches')37 ->method('getAllScheduledBatches')35 ->willReturnCallback(fn () => $this->scheduledBatches);38 ->willReturnCallback(fn () => $this->scheduledBatches);363937-$batchStatusManager->expects($this->atLeast(2))40+$batchStatusManager->expects($this->exactly(14))38 ->method('isBatchStillInProgress')41 ->method('isBatchStillInProgress')39 ->willReturnCallback(function ($batchName) {42 ->willReturnCallback(function ($batchName) {40return $this->batchesInProgress[$batchName];43return $this->batchesInProgress[$batchName];@@ -45,13 +48,13 @@ public function testTargetStaleChunksFirst(): void45 * I.e. 100 items in chunks of 10 will produce 10 iterations,48 * I.e. 100 items in chunks of 10 will produce 10 iterations,46 * full batch of 250 ill be processed in 5 chunks of 50 items49 * full batch of 250 ill be processed in 5 chunks of 50 items47 */50 */48-$z = (int) floor(TargetEntitiesSelector::ES_CHUNK_SIZE / TargetEntitiesSelector::SMALL_CHUNK_SIZE);51+$z = (int) floor($mainChunkSize / $staleChunkSize);495250-$batchStatusManager->expects($this->atLeast($z))53+$batchStatusManager->expects($this->exactly($z + 1))51 ->method('extractEntityIdsFromBatch')54 ->method('extractEntityIdsFromBatch')52- ->with('scheduled-batch-2', TargetEntitiesSelector::SMALL_CHUNK_SIZE)55+ ->with('scheduled-batch-2', $staleChunkSize)53- ->willReturnCallback(function () use (&$scheduledEntityIds) {56+ ->willReturnCallback(function () use (&$scheduledEntityIds, $staleChunkSize) {54-return array_splice($scheduledEntityIds, 0, TargetEntitiesSelector::SMALL_CHUNK_SIZE);57+return array_splice($scheduledEntityIds, 0, $staleChunkSize);55 });58 });565957 Redis::shouldReceive('get')60 Redis::shouldReceive('get')@@ -63,15 +66,15 @@ public function testTargetStaleChunksFirst(): void63$highPriorityEnd = 5030;66$highPriorityEnd = 5030;64$highPriorityRange = range($highPriorityStart, $highPriorityEnd);67$highPriorityRange = range($highPriorityStart, $highPriorityEnd);656866-$normalPrioritySize = TargetEntitiesSelector::ES_CHUNK_SIZE - count($highPriorityRange);69+$normalPrioritySize = $mainChunkSize - count($highPriorityRange);677068$normalPriorityStart = 1;71$normalPriorityStart = 1;69$normalPriorityEnd = 50;72$normalPriorityEnd = 50;70$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);73$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);717472 Redis::shouldReceive('spop')75 Redis::shouldReceive('spop')73 ->once()76 ->once()74- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)77+ ->with('activities-for-update-priority', $mainChunkSize)75 ->andReturn($highPriorityRange);78 ->andReturn($highPriorityRange);767977 Redis::shouldReceive('spop')80 Redis::shouldReceive('spop')@@ -87,8 +90,8 @@ public function testTargetStaleChunksFirst(): void87$normalPriority = $selectionList->getNormalPriorityEntityId();90$normalPriority = $selectionList->getNormalPriorityEntityId();88$highPriority = $selectionList->getHighPriorityEntityIds();91$highPriority = $selectionList->getHighPriorityEntityIds();899290-$startOffsetId = 101 + (($i - 1) * TargetEntitiesSelector::SMALL_CHUNK_SIZE);93+$startOffsetId = 101 + (($i - 1) * $staleChunkSize);91-$endOffsetId = 100 + ($i * TargetEntitiesSelector::SMALL_CHUNK_SIZE);94+$endOffsetId = 100 + ($i * $staleChunkSize);929593if ($i < ($z + 1)) {96if ($i < ($z + 1)) {94/**97/**@@ -126,8 +129,10 @@ public function testTargetStaleChunksFirst(): void126129127public function testOnlyHighPriorityEntitiesToProcess(): void130public function testOnlyHighPriorityEntitiesToProcess(): void128 {131 {132+$mainChunkSize = 200;133+129$batchStatusManager = $this->createMock(BatchStatusInterface::class);134$batchStatusManager = $this->createMock(BatchStatusInterface::class);130-$batchStatusManager->expects($this->atLeastOnce())135+$batchStatusManager->expects($this->once())131 ->method('getAllScheduledBatches')136 ->method('getAllScheduledBatches')132 ->willReturn([]);137 ->willReturn([]);133138@@ -137,12 +142,12 @@ public function testOnlyHighPriorityEntitiesToProcess(): void137142138 Redis::shouldReceive('spop')143 Redis::shouldReceive('spop')139 ->never()144 ->never()140- ->with('activities-for-update', TargetEntitiesSelector::ES_CHUNK_SIZE);145+ ->with('activities-for-update', $mainChunkSize);141146142 Redis::shouldReceive('spop')147 Redis::shouldReceive('spop')143 ->once()148 ->once()144- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)149+ ->with('activities-for-update-priority', $mainChunkSize)145- ->andReturn(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE));150+ ->andReturn(range(1, $mainChunkSize));146151147$selector = $this->getSelector($batchStatusManager);152$selector = $this->getSelector($batchStatusManager);148153@@ -153,7 +158,7 @@ public function testOnlyHighPriorityEntitiesToProcess(): void153$this->assertEmpty($normalPriority);158$this->assertEmpty($normalPriority);154$this->assertNotEmpty($highPriority);159$this->assertNotEmpty($highPriority);155160156-$this->assertEquals(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE), $highPriority);161+$this->assertEquals(range(1, $mainChunkSize), $highPriority);157 }162 }158163159private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector164private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY 20893 chunk control per update target by Vasil-</tabTitle>” with “<selection>@@ -20,6 +20,9 @@ class TargetEntitiesSelectorTest extends TestCase202021public function testTargetStaleChunksFirst(): void21public function testTargetStaleChunksFirst(): void22 {22 {23+$mainChunkSize = 200;24+$staleChunkSize = 50;25+23$batchStatusManager = $this->createMock(BatchStatusInterface::class);26$batchStatusManager = $this->createMock(BatchStatusInterface::class);242725$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];28$this->scheduledBatches = ['scheduled-batch-1', 'scheduled-batch-2'];@@ -28,13 +31,13 @@ public function testTargetStaleChunksFirst(): void28'scheduled-batch-2' => false,31'scheduled-batch-2' => false,29 ];32 ];30// Exactly one batch33// Exactly one batch31-$scheduledEntityIds = range(101, TargetEntitiesSelector::ES_CHUNK_SIZE + 100);34+$scheduledEntityIds = range(101, $mainChunkSize + 100);323533-$batchStatusManager->expects($this->atLeastOnce())36+$batchStatusManager->expects($this->exactly(5))34 ->method('getAllScheduledBatches')37 ->method('getAllScheduledBatches')35 ->willReturnCallback(fn () => $this->scheduledBatches);38 ->willReturnCallback(fn () => $this->scheduledBatches);363937-$batchStatusManager->expects($this->atLeast(2))40+$batchStatusManager->expects($this->exactly(14))38 ->method('isBatchStillInProgress')41 ->method('isBatchStillInProgress')39 ->willReturnCallback(function ($batchName) {42 ->willReturnCallback(function ($batchName) {40return $this->batchesInProgress[$batchName];43return $this->batchesInProgress[$batchName];@@ -45,13 +48,13 @@ public function testTargetStaleChunksFirst(): void45 * I.e. 100 items in chunks of 10 will produce 10 iterations,48 * I.e. 100 items in chunks of 10 will produce 10 iterations,46 * full batch of 250 ill be processed in 5 chunks of 50 items49 * full batch of 250 ill be processed in 5 chunks of 50 items47 */50 */48-$z = (int) floor(TargetEntitiesSelector::ES_CHUNK_SIZE / TargetEntitiesSelector::SMALL_CHUNK_SIZE);51+$z = (int) floor($mainChunkSize / $staleChunkSize);495250-$batchStatusManager->expects($this->atLeast($z))53+$batchStatusManager->expects($this->exactly($z + 1))51 ->method('extractEntityIdsFromBatch')54 ->method('extractEntityIdsFromBatch')52- ->with('scheduled-batch-2', TargetEntitiesSelector::SMALL_CHUNK_SIZE)55+ ->with('scheduled-batch-2', $staleChunkSize)53- ->willReturnCallback(function () use (&$scheduledEntityIds) {56+ ->willReturnCallback(function () use (&$scheduledEntityIds, $staleChunkSize) {54-return array_splice($scheduledEntityIds, 0, TargetEntitiesSelector::SMALL_CHUNK_SIZE);57+return array_splice($scheduledEntityIds, 0, $staleChunkSize);55 });58 });565957 Redis::shouldReceive('get')60 Redis::shouldReceive('get')@@ -63,15 +66,15 @@ public function testTargetStaleChunksFirst(): void63$highPriorityEnd = 5030;66$highPriorityEnd = 5030;64$highPriorityRange = range($highPriorityStart, $highPriorityEnd);67$highPriorityRange = range($highPriorityStart, $highPriorityEnd);656866-$normalPrioritySize = TargetEntitiesSelector::ES_CHUNK_SIZE - count($highPriorityRange);69+$normalPrioritySize = $mainChunkSize - count($highPriorityRange);677068$normalPriorityStart = 1;71$normalPriorityStart = 1;69$normalPriorityEnd = 50;72$normalPriorityEnd = 50;70$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);73$normalPriorityRange = range($normalPriorityStart, $normalPriorityEnd);717472 Redis::shouldReceive('spop')75 Redis::shouldReceive('spop')73 ->once()76 ->once()74- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)77+ ->with('activities-for-update-priority', $mainChunkSize)75 ->andReturn($highPriorityRange);78 ->andReturn($highPriorityRange);767977 Redis::shouldReceive('spop')80 Redis::shouldReceive('spop')@@ -87,8 +90,8 @@ public function testTargetStaleChunksFirst(): void87$normalPriority = $selectionList->getNormalPriorityEntityId();90$normalPriority = $selectionList->getNormalPriorityEntityId();88$highPriority = $selectionList->getHighPriorityEntityIds();91$highPriority = $selectionList->getHighPriorityEntityIds();899290-$startOffsetId = 101 + (($i - 1) * TargetEntitiesSelector::SMALL_CHUNK_SIZE);93+$startOffsetId = 101 + (($i - 1) * $staleChunkSize);91-$endOffsetId = 100 + ($i * TargetEntitiesSelector::SMALL_CHUNK_SIZE);94+$endOffsetId = 100 + ($i * $staleChunkSize);929593if ($i < ($z + 1)) {96if ($i < ($z + 1)) {94/**97/**@@ -126,8 +129,10 @@ public function testTargetStaleChunksFirst(): void126129127public function testOnlyHighPriorityEntitiesToProcess(): void130public function testOnlyHighPriorityEntitiesToProcess(): void128 {131 {132+$mainChunkSize = 200;133+129$batchStatusManager = $this->createMock(BatchStatusInterface::class);134$batchStatusManager = $this->createMock(BatchStatusInterface::class);130-$batchStatusManager->expects($this->atLeastOnce())135+$batchStatusManager->expects($this->once())131 ->method('getAllScheduledBatches')136 ->method('getAllScheduledBatches')132 ->willReturn([]);137 ->willReturn([]);133138@@ -137,12 +142,12 @@ public function testOnlyHighPriorityEntitiesToProcess(): void137142138 Redis::shouldReceive('spop')143 Redis::shouldReceive('spop')139 ->never()144 ->never()140- ->with('activities-for-update', TargetEntitiesSelector::ES_CHUNK_SIZE);145+ ->with('activities-for-update', $mainChunkSize);141146142 Redis::shouldReceive('spop')147 Redis::shouldReceive('spop')143 ->once()148 ->once()144- ->with('activities-for-update-priority', TargetEntitiesSelector::ES_CHUNK_SIZE)149+ ->with('activities-for-update-priority', $mainChunkSize)145- ->andReturn(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE));150+ ->andReturn(range(1, $mainChunkSize));146151147$selector = $this->getSelector($batchStatusManager);152$selector = $this->getSelector($batchStatusManager);148153@@ -153,7 +158,7 @@ public function testOnlyHighPriorityEntitiesToProcess(): void153$this->assertEmpty($normalPriority);158$this->assertEmpty($normalPriority);154$this->assertNotEmpty($highPriority);159$this->assertNotEmpty($highPriority);155160156-$this->assertEquals(range(1, TargetEntitiesSelector::ES_CHUNK_SIZE), $highPriority);161+$this->assertEquals(range(1, $mainChunkSize), $highPriority);157 }162 }158163159private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector164private function getSelector(BatchStatusInterface $batchStatusManager): TargetEntitiesSelector</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Summary of Changes
Summary of Changes
The selected diff updates
TargetEntitiesSelectorTest
by replacing hardcoded class constants with local variables (
$mainChunkSize
and
$staleChunkSize
) to make the tests more flexible, while explicitly tightening the expected call counts on mock objects.
Detailed Modifications
Detailed Modifications
1. Test Parameterization
1. Test Parameterization
Introduced Local Variables:
Replaced references to
TargetEntitiesSelector::ES_CHUNK_SIZE
with
$mainChunkSize = 200
and
TargetEntitiesSelector::SMALL_CHUNK_SIZE
with
$staleChunkSize = 50
.
Dynamic Range and Offset Calculations:
Updated array ranges, Redis
spop
arguments, and iteration offsets to derive dynamically from these local variables instead of hardcoded class constants.
2. Mock Expectation Hardening
2. Mock Expectation Hardening
getAllScheduledBatches
:
Changed expectations from
atLeastOnce()
to explicit strict counts:
exactly(5)
in
testTargetStaleChunksFirst
once()
in
testOnlyHighPriorityEntitiesToProcess
isBatchStillInProgress
:
Changed expectation from
atLeast(2)
to
exactly(14)
.
extractEntityIdsFromBatch
:
Changed expectation from
atLeast($z)
to
exactly($z + 1)
.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086/changes
|
50394
|
|
50547
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have no unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (29)...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086#pullrequestrevie github.com/jiminny/app/pull/12086#pullrequestreview-4308125516...
|
50547
|
|
50548
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have no unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086#pullrequestrevie github.com/jiminny/app/pull/12086#pullrequestreview-4308125516...
|
50548
|
|
50549
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have no unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (29)
Pull requests
(
29
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY 20893 chunk control per update target #12086 Edit title
JY 20893 chunk control per update target
#
12086
Edit title
Checks pending
Checks pending
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 18 commits into
master
master
from
JY-20893-chunk-control-per-update-target
JY-20893-chunk-control-per-update-target
Copy head branch name to clipboard
Lines changed: 158 additions & 35 deletions
Conversation
Conversation
Commits (18)
Commits
(
18
)
Checks
Checks
Files changed
Files changed
Open
JY 20893 chunk control per update target #12086 Vasil-Jiminny wants to merge 18 commits into master from JY-20893-chunk-control-per-update-target Copy head branch name to clipboard
JY 20893 chunk control per update target
JY 20893 chunk control per update target
#
12086
Vasil-Jiminny
Vasil-Jiminny
wants to merge 18 commits into
master
master
from
JY-20893-chunk-control-per-update-target
JY-20893-chunk-control-per-update-target
Copy head branch name to clipboard
Conversation
Conversation
@Vasil-Jiminny
Show options
Vasil-Jiminny commented 3 days ago •
Vasil-Jiminny
Vasil-Jiminny
commented
3 days ago
3 days ago
•...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086#pullrequestrevie github.com/jiminny/app/pull/12086#pullrequestreview-4308125516...
|
50549
|
|
50562
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
Jy 20846 mcp enable the ai to know details about the user by nikolaybiaivanov · Pull Request #12075 · jiminny/app
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (29)
Pull requests
(
29
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY 20893 chunk control per update target #12086 Edit title
JY 20893 chunk control per update target
#
12086
Edit title
Checks pending
Checks pending
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 18 commits into
master
master
from
JY-20893-chunk-control-per-update-target
JY-20893-chunk-control-per-update-target
Copy head branch name to clipboard
Lines changed: 158 additions & 35 deletions
Conversation (15)
Conversation
(
15
)
Commits (18)
Commits
(
18
)
Checks (3)
Checks
(
3
)
Files changed (6)
Files changed
(
6
)
Open
JY 20893 chunk control per update target #12086 Vasil-Jiminny wants to merge 18 commits into master from JY-20893-chunk-control-per-update-target Copy head branch name to clipboard
JY 20893 chunk control per update target
JY 20893 chunk control per update target
#
12086
Vasil-Jiminny
Vasil-Jiminny
wants to merge 18 commits into
master
master
from...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086#pullrequestrevie github.com/jiminny/app/pull/12086#pullrequestreview-4308125516...
|
50562
|
|
50563
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086#pullrequestrevie github.com/jiminny/app/pull/12086#pullrequestreview-4308125516...
|
50563
|
|
50203
|
pipedrive/client-php: Pipedrive API client for PHP pipedrive/client-php: Pipedrive API client for PHP
github.com
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
Close tab
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
github.com/jiminny/app/pull/12086
github.com/jiminny/app/pull/12086
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Waiting for github.com…...
|
Firefox
|
Work — Mozilla Firefox
|
github.com/jiminny/app/pull/12086
|
50203
|
|
50211
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50211
|
|
50212
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50212
|
|
50213
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (29)
Pull requests
(
29
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Review requested
Review requested
Vasil-Jiminny
Vasil-Jiminny
requested your review on this pull request.
Add your review
Add your review
JY 20893 chunk control per update target #12086 Edit title
JY 20893 chunk control per update target
#
12086
Edit title
Checks pending
Checks pending
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 17 commits into
master
master
from
JY-20893-chunk-control-per-update-target
JY-20893-chunk-control-per-update-target
Copy head branch name to clipboard
Lines changed: 158 additions & 35 deletions
Conversation (15)
Conversation
(
15
)
Commits (17)
Commits
(
17
)
Checks (2)
Checks
(
2
)
Files changed (6)
Files changed
(
6
)
Conversation
Conversation
@Vasil-Jiminny
Show options
Vasil-Jiminny commented 3 days ago •
Vasil-Jiminny
Vasil-Jiminny
commented
3 days ago
3 days ago
•
edited
edited
JIRA: JY-20893
JIRA:
JY-20893
JY-20893
Changes:
Changes:
Add a dedicated class to keep the chunk sizes for various updated targets for ES.
Update user classes and tests.
Drop obsolete constants.
Chunking algorithm tested with full local reset - works as expected.
Add or remove reactions
Vasil-Jiminny
Vasil-Jiminny
added
4
commits
3 days ago
3 days ago
@Vasil-Jiminny
Introduce ChunkSize to control the size of a chunk per update target.
Introduce ChunkSize to control the size of a chunk per update target.
731bdb6
731bdb6
@Vasil-Jiminny
SelectionListTest uses ChunkSize.
SelectionListTest uses ChunkSize.
6cda5d1
6cda5d1
@Vasil-Jiminny
SelectionList drop reference to TargetEntitiesSelector::ES_CHUNK_SIZE
SelectionList drop reference to
TargetEntitiesSelector::ES_CHUNK_SIZE
52848f5
52848f5
@Vasil-Jiminny
Update TargetEntitiesSelector to use ChunkSize instead of internal co…
Update TargetEntitiesSelector to use ChunkSize instead of internal co…
Commit message body
8 / 10 checks OK
f32fb68
f32fb68
@Vasil-Jiminny
Show options
Author
Vasil-Jiminny commented 3 days ago
Vasil-Jiminny
Vasil-Jiminny
commented
3 days ago
3 days ago
@claude
@claude
Add or remove reactions
react with eyes
👀
1
@claude
This comment was marked as outdated. Show comment
This comment was marked as outdated.
This comment was marked as outdated.
Show comment
@Vasil-Jiminny
Remove an unused import.
Remove an unused import.
8715e69
8715e69
@Vasil-Jiminny
Show options
Author
Vasil-Jiminny commented 3 days ago
Vasil-Jiminny
Vasil-Jiminny
commented
3 days ago
3 days ago
@claude
@claude
The changes in this PR are part of a series of PRs. Currently the SelectionList class requires something to provide a default value for the chunk size comparison. Within the next revision this class will be revised, and the constructor will be removed entirely in favor of accessor methods, including one about chunk size.
Rules such as adding comments, or using final are mostly dedicated to code generated by a model. The current changes are done by me (as part of a larger functional change), so they are okay.
Currently we use actively only one update targets - activity. Opportunity will have support added soon, hence why it is in the allowed list. All other options are currently not supported, and actively planned, but were intentionally added as potential candidates for future developments, if deemed necesary.
Add or remove reactions
react with eyes
👀
1
@Vasil-Jiminny
Remove comments.
Remove comments.
8 / 10 checks OK
36ce6e8
36ce6e8
@claude
This comment was marked as outdated. Show comment
This comment was marked as outdated.
This comment was marked as outdated.
Show comment
Vasil-Jiminny
Vasil-Jiminny
added
2
commits
3 days ago
3 days ago...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50213
|
|
50257
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
Edit
You said mysql update query
You said
mysql update query
Listen
Gemini said
Gemini said
The
UPDATE...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50257
|
|
50258
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
Edit
You said mysql update query
You said
mysql update query
Listen
Gemini said
Gemini said
The
UPDATE
statement in MySQL is used to modify existing records in a table.
Here is a complete guide to how it works, from basic usage to safer practices....
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50258
|
|
50259
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
Edit
You said mysql update query
You said
mysql update query
Listen
Gemini said
Gemini said
The
UPDATE
statement in MySQL is used to modify existing records in a table.
Here is a complete guide to how it works, from basic usage to safer practices.
1. The Basic Syntax
1. The Basic Syntax
The most critical part of an
UPDATE
query is the
WHERE
clause. If you omit it,
every single row
in your table will be updated.
SQL
Download code
Copy code
UPDATE table_name
SET
column1
=
value1, column2
=
value2, ...
WHERE
condition
;
2. Common Examples
2. Common Examples
Example A: Updating a single column for a specific row
Example A: Updating a single column for a specific row
Let's say you want to update the email address of an employee whose ID is
105
.
SQL
Download code
Copy code
UPDATE employees
SET
email
=
'[EMAIL]'
WHERE
employee_id
=
105
;
Example B: Updating multiple columns
Example B: Updating multiple columns
You can change multiple values at once by separating the
column = value
pairs with commas.
SQL
Download code
Copy code
UPDATE employees
SET
role
=
'Senior Developer'
, salary
=
95000
WHERE
employee_id
=
105
;
Example C: Using math or expressions
Example C: Using math or expressions
You can update a value based on its current value. For example, giving a 10% raise to everyone in department
4...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50259
|
|
50260
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
Edit
You said mysql update query
You said
mysql update query
Listen
Gemini said
Gemini said
The
UPDATE
statement in MySQL is used to modify existing records in a table.
Here is a complete guide to how it works, from basic usage to safer practices.
1. The Basic Syntax
1. The Basic Syntax
The most critical part of an
UPDATE
query is the
WHERE
clause. If you omit it,
every single row
in your table will be updated.
SQL
Download code
Copy code
UPDATE table_name
SET
column1
=
value1, column2
=
value2, ...
WHERE
condition
;
2. Common Examples
2. Common Examples
Example A: Updating a single column for a specific row
Example A: Updating a single column for a specific row
Let's say you want to update the email address of an employee whose ID is
105
.
SQL
Download code
Copy code
UPDATE employees
SET
email
=
'[EMAIL]'
WHERE
employee_id
=
105
;
Example B: Updating multiple columns
Example B: Updating multiple columns
You can change multiple values at once by separating the
column = value
pairs with commas.
SQL
Download code
Copy code
UPDATE employees
SET
role
=
'Senior Developer'
, salary
=
95000
WHERE
employee_id
=
105
;
Example C: Using math or expressions
Example C: Using math or expressions
You can update a value based on its current value. For example, giving a 10% raise to everyone in department
4
:
SQL
Download code
Copy code
UPDATE employees
SET
salary
=
salary
*
1.10
WHERE
department_id
=
4
;
3. Advanced Use Cases
3. Advanced Use Cases
Updating with a JOIN
Updating with a
JOIN
Sometimes you need to update a table based on values inside a completely different table.
SQL
Download code
Copy code
UPDATE employees e
JOIN
departments d
ON
e.department_id
=
d.id
SET
e.location
=
d.headquarters_city
WHERE
d.region
=
'North America'
;
Using LIMIT (Safety Measure)
Using
LIMIT
(Safety Measure)
To ensure you don't accidentally alter thousands of rows if your
WHERE
clause is too broad, you can append a
LIMIT
to restrict the maximum number of rows affected....
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50260
|
|
50261
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
JY-20891 add support for secondary email by LakyLak · Pull Request #12073 · jiminny/app
[JY-20891] Sidekick SMS issue - Jira
[JY-20891] Sidekick SMS issue - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
Usage | Windsurf
Usage | Windsurf
[SRD-6853] Moxso - Potential deal stages bug - Jira
[SRD-6853] Moxso - Potential deal stages bug - Jira
Pipelines - jiminny/app
Pipelines - jiminny/app
Feed — jiminny — Sentry
Feed — jiminny — Sentry
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
Pipedrive API Reference and Documentation
Pipedrive API Reference and Documentation
pipedrive/client-php: Pipedrive API client for PHP
pipedrive/client-php: Pipedrive API client for PHP
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
[jiminny/infrastructure] JY-20623 Add SQS queue for panorama reports (PR #728) - [EMAIL] - Jiminny Mail
Pull requests · jiminny/app
Pull requests · jiminny/app
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20912] Fallback mechanism for users with active SF tokens for CRM Matching - Jira
[JY-20906] Review of Pipedrive SDK - Jira
[JY-20906] Review of Pipedrive SDK - Jira
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app
Close tab
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
Jy 20918 remove redis switch for priority reindexing by Vasil-Jiminny · Pull Request #12088 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
Edit
You said mysql update query
You said
mysql update query
Listen
Gemini said
Gemini said
The
UPDATE
statement in MySQL is used to modify existing records in a table.
Here is a complete guide to how it works, from basic usage to safer practices.
1. The Basic Syntax
1. The Basic Syntax
The most critical part of an
UPDATE
query is the
WHERE
clause. If you omit it,
every single row
in your table will be updated.
SQL
Download code
Copy code
UPDATE table_name
SET
column1
=
value1, column2
=
value2, ...
WHERE
condition
;
2. Common Examples
2. Common Examples
Example A: Updating a single column for a specific row
Example A: Updating a single column for a specific row
Let's say you want to update the email address of an employee whose ID is
105
.
SQL
Download code
Copy code
UPDATE employees
SET
email
=
'[EMAIL]'
WHERE
employee_id
=
105
;
Example B: Updating multiple columns
Example B: Updating multiple columns
You can change multiple values at once by separating the
column = value
pairs with commas.
SQL
Download code
Copy code
UPDATE employees
SET
role
=
'Senior Developer'
, salary
=
95000
WHERE
employee_id
=
105
;
Example C: Using math or expressions
Example C: Using math or expressions
You can update a value based on its current value. For example, giving a 10% raise to everyone in department
4
:
SQL
Download code
Copy code
UPDATE employees
SET
salary
=
salary
*
1.10
WHERE
department_id
=
4
;
3. Advanced Use Cases
3. Advanced Use Cases
Updating with a JOIN
Updating with a
JOIN
Sometimes you need to update a table based on values inside a completely different table.
SQL
Download code
Copy code
UPDATE employees e
JOIN
departments d
ON
e.department_id
=
d.id
SET
e.location
=
d.headquarters_city
WHERE
d.region
=...
|
Firefox
|
JY 20893 chunk control per update target by Vasil- JY 20893 chunk control per update target by Vasil-Jiminny · Pull Request #12086 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12086
|
50261
|
|
55693
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,
entityId: (int) $document->getId(),
entityId: (int) $document->getId(),
withPriority: true
priority: QueuePriorityEnum::HIGH,
);
);
}
}
app/Component/ES/Processor/DTOs/SelectionList.php
+54-17Lines changed: 54 additions & 17 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
namespace Jiminny\Component\ES\Processor\DTOs;
namespace Jiminny\Component\ES\Processor\DTOs;
use Jiminny\Component\ES\QueuePriorityEnum;
/**
/**
* A very basic and simple collection of entity IDs.
* A very basic and simple collection of entity IDs.
*/
*/
class SelectionList
class SelectionList
{
{
public function __construct(
private const int DEFAULT_CHUNK = 200;
private readonly array $priorityEntityIdList = [],
private readonly array $normalEntityIdList = [],
private int $chunkSize = self::DEFAULT_CHUNK;
private readonly bool $isStaleChunkSelected = false,
private bool $isStaleChunkSelected = false;
private readonly int $chunkSize = 200,
private array $selectedEntityIds = [];
) {
// Do nothing
public function setChunkSize(int $chunkSize): void
{
$this->chunkSize = $chunkSize;
}
}
public function getAllIds(): array
public function markStale(): void
{
{
return array_merge($this->priorityEntityIdList, $this->normalEntityIdList);
$this->isStaleChunkSelected = true;
}
}
public function count(): int
public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void
{
{
return count($this->priorityEntityIdList) + count($this->normalEntityIdList);
$priorityKey = $this->normalisePriority($priority);
if (! array_key_exists($priorityKey, $this->selectedEntityIds)) {
$this->selectedEntityIds[$priorityKey] = [];
}
$this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds);
}
}
public function isEmpty(): bool
/**
* @return string[]
*/
public function getEntityListGroups(): array
{
{
return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList);
return array_keys($this->selectedEntityIds);
}
}
public function getHighPriorityEntityIds(): array
public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array
{
{
return $this->priorityEntityIdList;
$priorityKey = $this->normalisePriority($priority);
return $this->selectedEntityIds[$priorityKey] ?? [];
}
}
public function getNormalPriorityEntityId(): array
public function getAllIds(): array
{
{
return $this->normalEntityIdList;...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-eea github.com/jiminny/app/pull/12082/changes#diff-eeacd403a912dc6a384056b3fd5356e90255c2794f5665a40bbda5c683098397...
|
55693
|
|
55698
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,
entityId: (int) $document->getId(),
entityId: (int) $document->getId(),
withPriority: true
priority: QueuePriorityEnum::HIGH,
);
);
}
}
app/Component/ES/Processor/DTOs/SelectionList.php
+54-17Lines changed: 54 additions & 17 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
namespace Jiminny\Component\ES\Processor\DTOs;
namespace Jiminny\Component\ES\Processor\DTOs;
use Jiminny\Component\ES\QueuePriorityEnum;
/**
/**
* A very basic and simple collection of entity IDs.
* A very basic and simple collection of entity IDs.
*/
*/
class SelectionList
class SelectionList
{
{
public function __construct(
private const int DEFAULT_CHUNK = 200;
private readonly array $priorityEntityIdList = [],
private readonly array $normalEntityIdList = [],
private int $chunkSize = self::DEFAULT_CHUNK;
private readonly bool $isStaleChunkSelected = false,
private bool $isStaleChunkSelected = false;
private readonly int $chunkSize = 200,
private array $selectedEntityIds = [];
) {
// Do nothing
public function setChunkSize(int $chunkSize): void
{
$this->chunkSize = $chunkSize;
}
}
public function getAllIds(): array
public function markStale(): void
{
{
return array_merge($this->priorityEntityIdList, $this->normalEntityIdList);
$this->isStaleChunkSelected = true;
}
}
public function count(): int
public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void
{
{
return count($this->priorityEntityIdList) + count($this->normalEntityIdList);
$priorityKey = $this->normalisePriority($priority);
if (! array_key_exists($priorityKey, $this->selectedEntityIds)) {
$this->selectedEntityIds[$priorityKey] = [];
}
$this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds);
}
}
public function isEmpty(): bool
/**
* @return string[]
*/
public function getEntityListGroups(): array
{
{
return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList);
return array_keys($this->selectedEntityIds);
}
}
public function getHighPriorityEntityIds(): array
public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array
{
{
return $this->priorityEntityIdList;
$priorityKey = $this->normalisePriority($priority);
return $this->selectedEntityIds[$priorityKey] ?? [];
}
}
public function getNormalPriorityEntityId(): array
public function getAllIds(): array
{
{
return $this->normalEntityIdList;
$allIds = [];
foreach ($this->selectedEntityIds as $entityIds) {
$allIds = array_merge($allIds, $entityIds);
}
return $allIds;...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-eea github.com/jiminny/app/pull/12082/changes#diff-eeacd403a912dc6a384056b3fd5356e90255c2794f5665a40bbda5c683098397...
|
55698
|
|
55695
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,
entityId: (int) $document->getId(),
entityId: (int) $document->getId(),
withPriority: true
priority: QueuePriorityEnum::HIGH,
);
);
}
}
app/Component/ES/Processor/DTOs/SelectionList.php
+54-17Lines changed: 54 additions & 17 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
namespace Jiminny\Component\ES\Processor\DTOs;
namespace Jiminny\Component\ES\Processor\DTOs;
use Jiminny\Component\ES\QueuePriorityEnum;
/**
/**
* A very basic and simple collection of entity IDs.
* A very basic and simple collection of entity IDs.
*/
*/
class SelectionList
class SelectionList
{
{
public function __construct(
private const int DEFAULT_CHUNK = 200;
private readonly array $priorityEntityIdList = [],
private readonly array $normalEntityIdList = [],
private int $chunkSize = self::DEFAULT_CHUNK;
private readonly bool $isStaleChunkSelected = false,
private bool $isStaleChunkSelected = false;
private readonly int $chunkSize = 200,
private array $selectedEntityIds = [];
) {
// Do nothing
public function setChunkSize(int $chunkSize): void
{
$this->chunkSize = $chunkSize;
}
}
public function getAllIds(): array
public function markStale(): void
{
{
return array_merge($this->priorityEntityIdList, $this->normalEntityIdList);
$this->isStaleChunkSelected = true;
}...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-e5d github.com/jiminny/app/pull/12082/changes#diff-e5dcbcff071d212d7289a3afae523499caf0603272c50b6da48a278910b9ca09...
|
55695
|
|
55703
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,
entityId: (int) $document->getId(),
entityId: (int) $document->getId(),
withPriority: true
priority: QueuePriorityEnum::HIGH,
);
);
}
}
app/Component/ES/Processor/DTOs/SelectionList.php
+54-17Lines changed: 54 additions & 17 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
namespace Jiminny\Component\ES\Processor\DTOs;
namespace Jiminny\Component\ES\Processor\DTOs;
use Jiminny\Component\ES\QueuePriorityEnum;
/**
/**
* A very basic and simple collection of entity IDs.
* A very basic and simple collection of entity IDs.
*/
*/
class SelectionList
class SelectionList
{
{
public function __construct(
private const int DEFAULT_CHUNK = 200;
private readonly array $priorityEntityIdList = [],
private readonly array $normalEntityIdList = [],
private int $chunkSize = self::DEFAULT_CHUNK;
private readonly bool $isStaleChunkSelected = false,
private bool $isStaleChunkSelected = false;
private readonly int $chunkSize = 200,
private array $selectedEntityIds = [];
) {
// Do nothing
public function setChunkSize(int $chunkSize): void
{
$this->chunkSize = $chunkSize;
}
}
public function getAllIds(): array
public function markStale(): void
{
{
return array_merge($this->priorityEntityIdList, $this->normalEntityIdList);
$this->isStaleChunkSelected = true;
}
}
public function count(): int
public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void
{
{
return count($this->priorityEntityIdList) + count($this->normalEntityIdList);
$priorityKey = $this->normalisePriority($priority);
if (! array_key_exists($priorityKey, $this->selectedEntityIds)) {
$this->selectedEntityIds[$priorityKey] = [];
}
$this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds);
}
}
public function isEmpty(): bool
/**
* @return string[]
*/
public function getEntityListGroups(): array
{
{
return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList);
return array_keys($this->selectedEntityIds);
}
}
public function getHighPriorityEntityIds(): array
public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array
{
{
return $this->priorityEntityIdList;
$priorityKey = $this->normalisePriority($priority);
return $this->selectedEntityIds[$priorityKey] ?? [];
}
}
public function getNormalPriorityEntityId(): array
public function getAllIds(): array
{
{
return $this->normalEntityIdList;
$allIds = [];
foreach ($this->selectedEntityIds as $entityIds) {
$allIds = array_merge($allIds, $entityIds);
}
return $allIds;
}
public function count(): int
{
return array_sum(
array_map('count', $this->selectedEntityIds)...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-616 github.com/jiminny/app/pull/12082/changes#diff-61654edeebd72388bddd97cafcac889a2d106f54bb484a719d4067878bb9b011...
|
55703
|
|
55704
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,
entityId: (int) $document->getId(),
entityId: (int) $document->getId(),
withPriority: true
priority: QueuePriorityEnum::HIGH,
);
);
}
}
app/Component/ES/Processor/DTOs/SelectionList.php
+54-17Lines changed: 54 additions & 17 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
namespace Jiminny\Component\ES\Processor\DTOs;
namespace Jiminny\Component\ES\Processor\DTOs;
use Jiminny\Component\ES\QueuePriorityEnum;
/**
/**
* A very basic and simple collection of entity IDs.
* A very basic and simple collection of entity IDs.
*/
*/
class SelectionList
class SelectionList
{
{
public function __construct(
private const int DEFAULT_CHUNK = 200;
private readonly array $priorityEntityIdList = [],
private readonly array $normalEntityIdList = [],
private int $chunkSize = self::DEFAULT_CHUNK;
private readonly bool $isStaleChunkSelected = false,
private bool $isStaleChunkSelected = false;
private readonly int $chunkSize = 200,
private array $selectedEntityIds = [];
) {
// Do nothing
public function setChunkSize(int $chunkSize): void
{
$this->chunkSize = $chunkSize;
}
}
public function getAllIds(): array
public function markStale(): void
{
{
return array_merge($this->priorityEntityIdList, $this->normalEntityIdList);
$this->isStaleChunkSelected = true;
}
}
public function count(): int
public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void
{
{
return count($this->priorityEntityIdList) + count($this->normalEntityIdList);
$priorityKey = $this->normalisePriority($priority);
if (! array_key_exists($priorityKey, $this->selectedEntityIds)) {
$this->selectedEntityIds[$priorityKey] = [];
}
$this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds);
}
}
public function isEmpty(): bool
/**
* @return string[]
*/
public function getEntityListGroups(): array
{
{
return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList);
return array_keys($this->selectedEntityIds);
}
}
public function getHighPriorityEntityIds(): array
public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array
{
{
return $this->priorityEntityIdList;
$priorityKey = $this->normalisePriority($priority);
return $this->selectedEntityIds[$priorityKey] ?? [];
}
}
public function getNormalPriorityEntityId(): array
public function getAllIds(): array
{
{
return $this->normalEntityIdList;
$allIds = [];
foreach ($this->selectedEntityIds as $entityIds) {
$allIds = array_merge($allIds, $entityIds);
}
return $allIds;
}
public function count(): int
{
return array_sum(
array_map('count', $this->selectedEntityIds)
);
}
public function isEmpty(): bool
{
return $this->count() === 0;
}
}...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-616 github.com/jiminny/app/pull/12082/changes#diff-61654edeebd72388bddd97cafcac889a2d106f54bb484a719d4067878bb9b011...
|
55704
|
|
55705
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,
entityId: (int) $document->getId(),
entityId: (int) $document->getId(),
withPriority: true
priority: QueuePriorityEnum::HIGH,
);
);
}
}
app/Component/ES/Processor/DTOs/SelectionList.php
+54-17Lines changed: 54 additions & 17 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
namespace Jiminny\Component\ES\Processor\DTOs;
namespace Jiminny\Component\ES\Processor\DTOs;
use Jiminny\Component\ES\QueuePriorityEnum;
/**
/**
* A very basic and simple collection of entity IDs.
* A very basic and simple collection of entity IDs.
*/
*/
class SelectionList
class SelectionList
{
{
public function __construct(
private const int DEFAULT_CHUNK = 200;
private readonly array $priorityEntityIdList = [],
private readonly array $normalEntityIdList = [],
private int $chunkSize = self::DEFAULT_CHUNK;
private readonly bool $isStaleChunkSelected = false,
private bool $isStaleChunkSelected = false;
private readonly int $chunkSize = 200,
private array $selectedEntityIds = [];
) {
// Do nothing
public function setChunkSize(int $chunkSize): void
{
$this->chunkSize = $chunkSize;
}
}
public function getAllIds(): array
public function markStale(): void
{
{
return array_merge($this->priorityEntityIdList, $this->normalEntityIdList);
$this->isStaleChunkSelected = true;
}
}...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-616 github.com/jiminny/app/pull/12082/changes#diff-61654edeebd72388bddd97cafcac889a2d106f54bb484a719d4067878bb9b011...
|
55705
|
|
55706
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,
entityId: (int) $document->getId(),
entityId: (int) $document->getId(),
withPriority: true
priority: QueuePriorityEnum::HIGH,
);
);
}
}
app/Component/ES/Processor/DTOs/SelectionList.php
+54-17Lines changed: 54 additions & 17 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
namespace Jiminny\Component\ES\Processor\DTOs;
namespace Jiminny\Component\ES\Processor\DTOs;
use Jiminny\Component\ES\QueuePriorityEnum;
/**
/**
* A very basic and simple collection of entity IDs.
* A very basic and simple collection of entity IDs.
*/
*/
class SelectionList
class SelectionList
{
{
public function __construct(
private const int DEFAULT_CHUNK = 200;
private readonly array $priorityEntityIdList = [],
private readonly array $normalEntityIdList = [],
private int $chunkSize = self::DEFAULT_CHUNK;
private readonly bool $isStaleChunkSelected = false,
private bool $isStaleChunkSelected = false;
private readonly int $chunkSize = 200,
private array $selectedEntityIds = [];
) {
// Do nothing
public function setChunkSize(int $chunkSize): void
{
$this->chunkSize = $chunkSize;
}
}
public function getAllIds(): array
public function markStale(): void
{
{
return array_merge($this->priorityEntityIdList, $this->normalEntityIdList);
$this->isStaleChunkSelected = true;
}
}
public function count(): int
public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void
{
{
return count($this->priorityEntityIdList) + count($this->normalEntityIdList);
$priorityKey = $this->normalisePriority($priority);
if (! array_key_exists($priorityKey, $this->selectedEntityIds)) {
$this->selectedEntityIds[$priorityKey] = [];
}
$this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds);
}
}
public function isEmpty(): bool
/**
* @return string[]
*/
public function getEntityListGroups(): array
{
{
return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList);
return array_keys($this->selectedEntityIds);
}
}
public function getHighPriorityEntityIds(): array
public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array
{
{
return $this->priorityEntityIdList;
$priorityKey = $this->normalisePriority($priority);
return $this->selectedEntityIds[$priorityKey] ?? [];
}
}
public function getNormalPriorityEntityId(): array
public function getAllIds(): array
{
{
return $this->normalEntityIdList;
$allIds = [];
foreach ($this->selectedEntityIds as $entityIds) {
$allIds = array_merge($allIds, $entityIds);
}
return $allIds;
}
public function count(): int
{
return array_sum(
array_map('count', $this->selectedEntityIds)
);
}
public function isEmpty(): bool
{
return $this->count() === 0;
}
}
public function isStaleChunkSelected(): bool
public function isStaleChunkSelected(): bool
{
{
return $this->count() >= $this->chunkSize;
return $this->count() >= $this->chunkSize;
}
}
private function normalisePriority(string|QueuePriorityEnum $priority): string
{
return $priority instanceof QueuePriorityEnum ? $priority->value : $priority;
}
}
}
app/Component/ES/Processor/Traits/SelectEntityListTrait.php
+27-4Lines changed: 27 additions & 4 deletions
Original file line numberOriginal file lineDiff line numberDiff line change
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Log;...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-616 github.com/jiminny/app/pull/12082/changes#diff-61654edeebd72388bddd97cafcac889a2d106f54bb484a719d4067878bb9b011...
|
55706
|
|
55719
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-616 github.com/jiminny/app/pull/12082/changes#diff-61654edeebd72388bddd97cafcac889a2d106f54bb484a719d4067878bb9b011...
|
55719
|
|
55656
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said what about non test files
You said
what about non test files
Listen
Show thinking
Gemini said
Gemini said
Since we don't have the actual
SelectionList.php
file, we have to reverse-engineer its implementation based on the test behaviors. Moving from a constructor-based, fixed-property model to a dynamic, mutable model introduces several potential pitfalls in the implementation file itself.
Here are the specific bottlenecks and issues to watch out for in the
SelectionList.php
source code:
1. The count() Method Implementation (Performance Trap)
1. The
count()
Method Implementation (Performance Trap)
The Risk:
If
count()
is implemented by calling
$this->getAllIds()
and counting the resulting merged array (e.g.,
return count($this->getAllIds());
), it will be highly inefficient. It would allocate a brand-new array in memory just to determine its length.
The Fix:
count()
should mathematically sum the sizes of the underlying group arrays without merging them:
return array_sum(array_map('count', $this->entityLists));
2. Loss of Immutability (State Management)
2. Loss of Immutability (State Management)...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-4bd github.com/jiminny/app/pull/12082/changes#diff-4bdd02759b5e0d060dcbc8c2dcb9bdbc3d0266d9c2e29956b9d63916dae1fdd7...
|
55656
|
|
55657
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said what about non test files
You said
what about non test files
Listen
Show thinking
Gemini said
Gemini said
Since we don't have the actual
SelectionList.php
file, we have to reverse-engineer its implementation based on the test behaviors. Moving from a constructor-based, fixed-property model to a dynamic, mutable model introduces several potential pitfalls in the implementation file itself.
Here are the specific bottlenecks and issues to watch out for in the
SelectionList.php
source code:
1. The count() Method Implementation (Performance Trap)
1. The
count()
Method Implementation (Performance Trap)
The Risk:
If
count()
is implemented by calling
$this->getAllIds()
and counting the resulting merged array (e.g.,
return count($this->getAllIds());
), it will be highly inefficient. It would allocate a brand-new array in memory just to determine its length.
The Fix:
count()
should mathematically sum the sizes of the underlying group arrays without merging them:
return array_sum(array_map('count', $this->entityLists));
2. Loss of Immutability (State Management)
2. Loss of Immutability (State Management)
The Risk:
The original code passed all data via the constructor, implying the DTO (Data Transfer Object) was immutable once created. The refactored version introduces mutability (
addEntityList()
,
setChunkSize()
,
markStale()
). DTOs are typically passed between architectural layers; if one layer modifies the chunk size or adds entities while another is processing it, it can cause unpredictable side effects or race conditions in your queue processor.
The Fix:
Ensure that this object is not shared across asynchronous contexts, or consider returning cloned instances when mutating state if strict immutability is required.
3. Enum Array Key Violations (PHP Quirks)
3. Enum Array Key Violations (PHP Quirks)
The Risk:
The tests show
QueuePriorityEnum::HIGH
being passed to methods. If the underlying code attempts to use this Enum directly as an array key (e.g.,
$this->lists[$priority] = $ids;
), PHP will throw an error because objects (including Enums) cannot be used as standard array keys.
The Fix:
The implementation must map the Enum to its underlying scalar value (e.g.,
$this->lists[$priority->value]
) or use PHP's
WeakMap
or
SplObjectStorage
, though standard arrays using
$priority->value
are much faster for simple lists.
4. Uninitialized State Defaults (Logic Errors)
4. Uninitialized State Defaults (Logic Errors)
The Risk:
The tests instantiate
new SelectionList()
without arguments, then later call
$selectionList->setChunkSize(100)
. If
isFullChunk()
is called
before
setChunkSize()
is invoked, what happens?
The Fix:
Ensure
SelectionList
has a sensible default for chunk size (e.g.,
private int $chunkSize = 0;
or a system default) to prevent uninitialized property errors or division-by-zero scenarios if your logic ever relies on chunk math.
5. getAllIds() Repeated Execution (CPU Overhead)
5.
getAllIds()
Repeated Execution (CPU Overhead)
The Risk:
If
getAllIds()
dynamically merges arrays (
array_merge(...$this->lists)
) every time it is invoked, and your ES processor calls this method inside a loop, it will spike CPU usage.
The Fix:
If the list is read frequently, the implementation should either cache the merged result and invalidate that cache when
addEntityList()
is called, or the caller should assign the result to a local variable before iterating.
Would you like me to draft an optimal, memory-efficient skeleton for the
SelectionList.php
implementation that avoids these specific pitfalls?
Good response
Bad response
Share & export
Copy
Show more options...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-4bd github.com/jiminny/app/pull/12082/changes#diff-4bdd02759b5e0d060dcbc8c2dcb9bdbc3d0266d9c2e29956b9d63916dae1fdd7...
|
55657
|
|
55658
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said what about non test files
You said
what about non test files
Listen
Show thinking
Gemini said
Gemini said
Since we don't have the actual
SelectionList.php
file, we have to reverse-engineer its implementation based on the test behaviors. Moving from a constructor-based, fixed-property model to a dynamic, mutable model introduces several potential pitfalls in the implementation file itself.
Here are the specific bottlenecks and issues to watch out for in the
SelectionList.php
source code:
1. The count() Method Implementation (Performance Trap)
1. The
count()
Method Implementation (Performance Trap)
The Risk:
If
count()
is implemented by calling
$this->getAllIds()
and counting the resulting merged array (e.g.,
return count($this->getAllIds());
), it will be highly inefficient. It would allocate a brand-new array in memory just to determine its length.
The Fix:
count()
should mathematically sum the sizes of the underlying group arrays without merging them:
return array_sum(array_map('count', $this->entityLists));
2. Loss of Immutability (State Management)
2. Loss of Immutability (State Management)
The Risk:
The original code passed all data via the constructor, implying the DTO (Data Transfer Object) was immutable once created. The refactored version introduces mutability (
addEntityList()
,
setChunkSize()
,
markStale()
). DTOs are typically passed between architectural layers; if one layer modifies the chunk size or adds entities while another is processing it, it can cause unpredictable side effects or race conditions in your queue processor.
The Fix:
Ensure that this object is not shared across asynchronous contexts, or consider returning cloned instances when mutating state if strict immutability is required.
3. Enum Array Key Violations (PHP Quirks)
3. Enum Array Key Violations (PHP Quirks)
The Risk:
The tests show
QueuePriorityEnum::HIGH
being passed to methods. If the underlying code attempts to use this Enum directly as an array key (e.g.,
$this->lists[$priority] = $ids;
), PHP will throw an error because objects (including Enums) cannot be used as standard array keys.
The Fix:
The implementation must map the Enum to its underlying scalar value (e.g.,
$this->lists[$priority->value]
) or use PHP's
WeakMap
or
SplObjectStorage
, though standard arrays using
$priority->value
are much faster for simple lists.
4. Uninitialized State Defaults (Logic Errors)
4. Uninitialized State Defaults (Logic Errors)
The Risk:
The tests instantiate
new SelectionList()
without arguments, then later call
$selectionList->setChunkSize(100)
. If
isFullChunk()
is called
before
setChunkSize()
is invoked, what happens?
The Fix:
Ensure
SelectionList
has a sensible default for chunk size (e.g.,
private int $chunkSize = 0;
or a system default) to prevent uninitialized property errors or division-by-zero scenarios if your logic ever relies on chunk math.
5. getAllIds() Repeated Execution (CPU Overhead)
5.
getAllIds()
Repeated Execution (CPU Overhead)
The Risk:
If
getAllIds()
dynamically merges arrays (
array_merge(...$this->lists)
) every time it is invoked, and your ES processor calls this method inside a loop, it will spike CPU usage.
The Fix:
If the list is read frequently, the implementation should either cache the merged result and invalidate that cache when
addEntityList()
is called, or the caller should assign the result to a local variable before iterating.
Would you like me to draft an optimal, memory-efficient skeleton for the
SelectionList.php
implementation that avoids these specific pitfalls?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said can't you the PR. How does it work when I click summarize page in gemini via firefox
You said
can't you the PR. How does it work when I click summarize page in gemini via firefox
Listen...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-4bd github.com/jiminny/app/pull/12082/changes#diff-4bdd02759b5e0d060dcbc8c2dcb9bdbc3d0266d9c2e29956b9d63916dae1fdd7...
|
55658
|
|
55659
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-4bd github.com/jiminny/app/pull/12082/changes#diff-4bdd02759b5e0d060dcbc8c2dcb9bdbc3d0266d9c2e29956b9d63916dae1fdd7...
|
55659
|
|
55660
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-4bd github.com/jiminny/app/pull/12082/changes#diff-4bdd02759b5e0d060dcbc8c2dcb9bdbc3d0266d9c2e29956b9d63916dae1fdd7...
|
55660
|
|
55720
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Jiminny\Contracts\ES\Events\UpdateMultipleEntities;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntities(),
$event->getEntities(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
}
}
app/Component/ES/Listeners/UpdateSingleTargetListener.php
+9-1Lines changed: 9 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\Actions\TrackElasticSearchOperations;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\AsyncUpdateElasticSearch;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\Processor\EntityQueryBuilder;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Jiminny\Contracts\ES\Events\UpdateSingleEntity;
use Psr\Log\LoggerInterface;
use Psr\Log\LoggerInterface;
'entityId' => $event->getEntityId(),
'entityId' => $event->getEntityId(),
]);
]);
/**
* @important:
* Synchronous event should eventually be deprecated.
* Instant write operations are blocking and should be used sparingly.
* It is much better in terms of performance to write 100 or 1000 documents at once,
* and have one blocking reindex operation, instead of writing 10 documents synchronously.
*/
if ($event->isSyncEvent()) {
if ($event->isSyncEvent()) {
$this->handleSyncEvent($event);
$this->handleSyncEvent($event);
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
$event->getUpdateTargetValue(),
$event->getUpdateTargetValue(),
$event->getEntityId(),
$event->getEntityId(),
$event->isPriority()
$event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL,
);
);
}
}
app/Component/ES/Processor/Actions/UpsertDocumentsAction.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\ResponseEvaluator;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;
use Jiminny\Component\ES\QueuePriorityEnum;
use Jiminny\Exceptions\InvalidDataException;
use Jiminny\Exceptions\InvalidDataException;
use Sentry\Laravel\Facade as Sentry;
use Sentry\Laravel\Facade as Sentry;
AsyncUpdateElasticSearch::markEntityForUpdate(
AsyncUpdateElasticSearch::markEntityForUpdate(
entityType: $this->updateTarget,
entityType: $this->updateTarget,...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-4bd github.com/jiminny/app/pull/12082/changes#diff-4bdd02759b5e0d060dcbc8c2dcb9bdbc3d0266d9c2e29956b9d63916dae1fdd7...
|
55720
|
|
55702
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said app/Component/ES/Listeners/UpdateMultipleTargetsListener.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners; namespace Jiminny\Component\ES\Listeners; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Jiminny\Contracts\ES\Events\UpdateMultipleEntities; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntities(), $event->getEntities(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } } } app/Component/ES/Listeners/UpdateSingleTargetListener.php +9-1Lines changed: 9 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\Actions\TrackElasticSearchOperations; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Jiminny\Contracts\ES\Events\UpdateSingleEntity; use Psr\Log\LoggerInterface; use Psr\Log\LoggerInterface; 'entityId' => $event->getEntityId(), 'entityId' => $event->getEntityId(), ]); ]); /** * @important: * Synchronous event should eventually be deprecated. * Instant write operations are blocking and should be used sparingly. * It is much better in terms of performance to write 100 or 1000 documents at once, * and have one blocking reindex operation, instead of writing 10 documents synchronously. */ if ($event->isSyncEvent()) { if ($event->isSyncEvent()) { $this->handleSyncEvent($event); $this->handleSyncEvent($event); AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( $event->getUpdateTargetValue(), $event->getUpdateTargetValue(), $event->getEntityId(), $event->getEntityId(), $event->isPriority() $event->isPriority() ? QueuePriorityEnum::HIGH : QueuePriorityEnum::NORMAL, ); ); } } app/Component/ES/Processor/Actions/UpsertDocumentsAction.php +2-1Lines changed: 2 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\DTOs\SimpleCollection; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\ResponseEvaluator; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidDataException; use Jiminny\Exceptions\InvalidDataException; use Sentry\Laravel\Facade as Sentry; use Sentry\Laravel\Facade as Sentry; AsyncUpdateElasticSearch::markEntityForUpdate( AsyncUpdateElasticSearch::markEntityForUpdate( entityType: $this->updateTarget, entityType: $this->updateTarget, entityId: (int) $document->getId(), entityId: (int) $document->getId(), withPriority: true priority: QueuePriorityEnum::HIGH, ); ); } } app/Component/ES/Processor/DTOs/SelectionList.php +54-17Lines changed: 54 additions & 17 deletions Original file line numberOriginal file lineDiff line numberDiff line change namespace Jiminny\Component\ES\Processor\DTOs; namespace Jiminny\Component\ES\Processor\DTOs; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * A very basic and simple collection of entity IDs. * A very basic and simple collection of entity IDs. */ */ class SelectionList class SelectionList { { public function __construct( private const int DEFAULT_CHUNK = 200; private readonly array $priorityEntityIdList = [], private readonly array $normalEntityIdList = [], private int $chunkSize = self::DEFAULT_CHUNK; private readonly bool $isStaleChunkSelected = false, private bool $isStaleChunkSelected = false; private readonly int $chunkSize = 200, private array $selectedEntityIds = []; ) { // Do nothing public function setChunkSize(int $chunkSize): void { $this->chunkSize = $chunkSize; } } public function getAllIds(): array public function markStale(): void { { return array_merge($this->priorityEntityIdList, $this->normalEntityIdList); $this->isStaleChunkSelected = true; } } public function count(): int public function addEntityList(string|QueuePriorityEnum $priority, array $entityIds): void { { return count($this->priorityEntityIdList) + count($this->normalEntityIdList); $priorityKey = $this->normalisePriority($priority); if (! array_key_exists($priorityKey, $this->selectedEntityIds)) { $this->selectedEntityIds[$priorityKey] = []; } $this->selectedEntityIds[$priorityKey] = array_merge($this->selectedEntityIds[$priorityKey], $entityIds); } } public function isEmpty(): bool /** * @return string[] */ public function getEntityListGroups(): array { { return empty($this->priorityEntityIdList) && empty($this->normalEntityIdList); return array_keys($this->selectedEntityIds); } } public function getHighPriorityEntityIds(): array public function getEntityListItemsByGroup(string|QueuePriorityEnum $priority): array { { return $this->priorityEntityIdList; $priorityKey = $this->normalisePriority($priority); return $this->selectedEntityIds[$priorityKey] ?? []; } } public function getNormalPriorityEntityId(): array public function getAllIds(): array { { return $this->normalEntityIdList; $allIds = []; foreach ($this->selectedEntityIds as $entityIds) { $allIds = array_merge($allIds, $entityIds); } return $allIds; } public function count(): int { return array_sum( array_map('count', $this->selectedEntityIds) ); } public function isEmpty(): bool { return $this->count() === 0; } } public function isStaleChunkSelected(): bool public function isStaleChunkSelected(): bool { { return $this->count() >= $this->chunkSize; return $this->count() >= $this->chunkSize; } } private function normalisePriority(string|QueuePriorityEnum $priority): string { return $priority instanceof QueuePriorityEnum ? $priority->value : $priority; } } } app/Component/ES/Processor/Traits/SelectEntityListTrait.php +27-4Lines changed: 27 additions & 4 deletions Original file line numberOriginal file lineDiff line numberDiff line change use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Log; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; /** /** * Build the correct source list name for the type of entity and priority * Build the correct source list name for the type of entity and priority */ */ trait SelectEntityListTrait trait SelectEntityListTrait { { /** * Highest priority queue for data requiring near-instant availability. * Very few entities should be added here, only when absolutely necessary. */ private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; /** * Normal priority queue for normal application operations. * Almost all update events should pass through this queue */ private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_NORMAL_PRIORITIES_LIST = '%s-for-update'; private const string ENTITIES_HIGH_PRIORITIES_LIST = '%s-for-update-priority'; protected static function selectTargetList(string $entityType, bool $withPriority = false): string /** { * Nonblocking low priority queue. This queue will be consumed only when there is nothing * with higher priority. * * This queue will be used for operations such as es:reset-async */ private const string ENTITIES_LOW_PRIORITIES_LIST = '%s-for-update-low'; protected static function selectTargetList( string $entityType, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { if (! in_array($entityType, UpdateTarget::allowedTargets())) { if (! in_array($entityType, UpdateTarget::allowedTargets())) { throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); throw new InvalidArgumentException('AsyncUpdateElasticSearch, invalid entity type'); } } if ($withPriority) { if ($priority->isHigh()) { Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); Log::debug('[AsyncUpdateElasticSearch] - Priority check passed and priority is on'); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_HIGH_PRIORITIES_LIST, $entityType); } } if ($priority->isLow()) { return sprintf(self::ENTITIES_LOW_PRIORITIES_LIST, $entityType); } return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); return sprintf(self::ENTITIES_NORMAL_PRIORITIES_LIST, $entityType); } } } } app/Component/ES/Processor/TargetEntitiesSelector.php +26-21Lines changed: 26 additions & 21 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\DTOs\SelectionList; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait; use Jiminny\Component\ES\QueuePriorityEnum; /** /** * This class supplies a list of entities pending an update. * This class supplies a list of entities pending an update. public function select(): SelectionList public function select(): SelectionList { { $staleWorkEntityIds = $this->getStaleChunk(); $selectionList = new SelectionList(); $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $staleWorkEntityIds = $this->getStaleChunk(); if (count($staleWorkEntityIds) > 0) { if (count($staleWorkEntityIds) > 0) { return new SelectionList( /** priorityEntityIdList: $staleWorkEntityIds, * Stale chunks are treated as high priority isStaleChunkSelected: true, */ chunkSize: ChunkSize::getStaleChunkSize($this->updateTarget), $selectionList->markStale(); ); $selectionList->addEntityList(QueuePriorityEnum::HIGH, $staleWorkEntityIds); $selectionList->setChunkSize(ChunkSize::getStaleChunkSize($this->updateTarget)); return $selectionList; } } $mainChunkSize = ChunkSize::getMainChunkSize($this->updateTarget); $selectionList->setChunkSize($mainChunkSize); $priorityEntityIdsList = $this->extractIdsFromSet( $priorityEntityIdsList = $this->extractIds(QueuePriorityEnum::HIGH, $mainChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, true), $selectionList->addEntityList(QueuePriorityEnum::HIGH, $priorityEntityIdsList); $mainChunkSize, ); $normalEntityIdsList = []; $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); $normalEntityChunkSize = $mainChunkSize - count($priorityEntityIdsList); if ($normalEntityChunkSize > 0) { if ($normalEntityChunkSize > 0) { $normalEntityIdsList = $this->extractIdsFromSet( $normalEntityIdsList = $this->extractIds(QueuePriorityEnum::NORMAL, $normalEntityChunkSize); AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, false), $selectionList->addEntityList(QueuePriorityEnum::NORMAL, $normalEntityIdsList); $normalEntityChunkSize ); } } return new SelectionList( return $selectionList; priorityEntityIdList: $priorityEntityIdsList, normalEntityIdList: $normalEntityIdsList, chunkSize: $mainChunkSize ); } } private function getStaleChunk(): array private function getStaleChunk(): array return []; return []; } } private function extractIds(QueuePriorityEnum $priority, int $chunkSize): array { return $this->extractIdsFromSet( AsyncUpdateElasticSearch::getEntitiesSourceList($this->updateTarget, $priority), $chunkSize ); } } } app/Component/ES/AsyncUpdateElasticSearch.php +17-10Lines changed: 17 additions & 10 deletions Original file line numberOriginal file lineDiff line numberDiff line change * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * Various methods and actions call this method to rebuild auto-score, plays, shares, topic triggers * and other activity related data * and other activity related data */ */ public static function markEntityForUpdate(string $entityType, int $entityId, bool $withPriority = false): void public static function markEntityForUpdate( { string $entityType, $targetList = self::selectTargetList($entityType, $withPriority); int $entityId, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { $targetList = self::selectTargetList($entityType, $priority); Redis::sadd($targetList, $entityId); Redis::sadd($targetList, $entityId); Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ Log::info('[ AsyncUpdateElasticSearch ] Entity added to Redis list', [ 'entityType' => $entityType, 'entityType' => $entityType, 'entityId' => $entityId, 'entityId' => $entityId, 'collectionKey' => $targetList, 'collectionKey' => $targetList, 'withPriority' => $withPriority, 'priority' => $priority->value, ]); ]); } } public static function bulkMarkEntitiesForUpdate( public static function bulkMarkEntitiesForUpdate( string $entityType, string $entityType, array $entityIds, array $entityIds, bool $withPriority = false QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): void { ): void { if (empty($entityIds)) { if (empty($entityIds)) { return; return; } } $targetList = self::selectTargetList($entityType, $withPriority); $targetList = self::selectTargetList($entityType, $priority); Redis::saddarray($targetList, $entityIds); Redis::saddarray($targetList, $entityIds); } } * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Expose the Redis list name, which holds the collection of scheduled entities to be processed. * Takes priority into consideration. * Takes priority into consideration. */ */ public static function getEntitiesSourceList(string $updateTarget, $priority = false): string public static function getEntitiesSourceList( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): string { return self::selectTargetList($updateTarget, $priority); return self::selectTargetList($updateTarget, $priority); } } /** /** * Externally called helper methods * Externally called helper methods * - count and flush * - count and flush */ */ public static function getEntitiesListCount(string $updateTarget, $priority = false): int public static function getEntitiesListCount( { string $updateTarget, QueuePriorityEnum $priority = QueuePriorityEnum::NORMAL ): int { $targetList = self::selectTargetList($updateTarget, $priority); $targetList = self::selectTargetList($updateTarget, $priority); return Redis::scard($targetList); return Redis::scard($targetList); app/Component/ES/QueuePriorityEnum.php +40Lines changed: 40 additions & 0 deletions Original file line numberOriginal file lineDiff line numberDiff line change <?php declare(strict_types=1); namespace Jiminny\Component\ES; use Jiminny\Exceptions\InvalidArgumentException; enum QueuePriorityEnum: string { case HIGH = 'high'; case NORMAL = 'normal'; case LOW = 'low'; public static function tryFromString(string $value): self { if (! self::isValid($value)) { throw new InvalidArgumentException( sprintf('Invalid queue priority "%s" selected', $value) ); } return self::tryFrom($value); } public static function isValid(string $value): bool { return self::tryFrom($value) !== null; } public function isLow(): bool { return $this === self::LOW; } public function isHigh(): bool { return $this === self::HIGH; } } app/Component/ES/UpdateProcessManager.php +12-9Lines changed: 12 additions & 9 deletions Original file line numberOriginal file lineDiff line numberDiff line change if ($selectionList->isEmpty()) { if ($selectionList->isEmpty()) { return false; return false; } } /** /** * A stale chunk will always be smaller than a regular chunk. * A stale chunk will always be smaller than a regular chunk. * That doesn't mean there aren't any other scheduled entities for processing * That doesn't mean there aren't any other scheduled entities for processing private function rescheduleThrottledEntities(SelectionList $selectionList): void private function rescheduleThrottledEntities(SelectionList $selectionList): void { { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $groups = $selectionList->getEntityListGroups(); $this->updateTarget, foreach ($groups as $eachGroup) { $selectionList->getNormalPriorityEntityId() $groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup); ); if (! empty($groupEntityIds)) { AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate( $this->updateTarget, entityType: $this->updateTarget, $selectionList->getHighPriorityEntityIds(), entityIds: $groupEntityIds, true priority: QueuePriorityEnum::tryFromString($eachGroup) ); ); } } } } } } app/Console/Commands/Elasticsearch/AsyncUpdateEsEntities.php +9-3Lines changed: 9 additions & 3 deletions Original file line numberOriginal file lineDiff line numberDiff line changeuse Illuminate\Support\Str; use Illuminate\Support\Str; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\AsyncUpdateElasticSearch; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Component\ES\UpdateProcessManager; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; Redis::set($gaugeLockName, true); Redis::set($gaugeLockName, true); Redis::expire($gaugeLockName, 60); Redis::expire($gaugeLockName, 60); $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::HIGH), ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update', $updateTarget), stat: sprintf('jiminny.%s-for-update', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::NORMAL), ); ); $this->gaugeStats( $this->gaugeStats( stat: sprintf('jiminny.%s-for-update-priority', $updateTarget), stat: sprintf('jiminny.%s-for-update-low', $updateTarget), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, true), value: AsyncUpdateElasticSearch::getEntitiesListCount($updateTarget, QueuePriorityEnum::LOW), ); ); } } app/Console/Commands/Elasticsearch/ResetAsyncElasticSearchCommand.php +5-1Lines changed: 5 additions & 1 deletion Original file line numberOriginal file lineDiff line numberDiff line changeuse Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\EntityQueryBuilder; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\Traits\ValidateUpdateTargetTrait; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\Processor\UpdateTarget; use Jiminny\Component\ES\QueuePriorityEnum; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Component\ES\Repositories\EsResetRepositoryInterface; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Exceptions\InvalidArgumentException; use Jiminny\Traits\GracefullyStoppable; use Jiminny\Traits\GracefullyStoppable; private function getRemainingScheduledEntities(bool $printStatus = false): int private function getRemainingScheduledEntities(bool $printStatus = false): int { { $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingRegular = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount($this->updateTarget, true); $remainingPriority = AsyncUpdateElasticSearch::getEntitiesListCount( $this->updateTarget, QueuePriorityEnum::HIGH ); if ($printStatus) { if ($printStatus) { $this->line( $this->line(
You said
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberOriginal file lineDiff line numberDiff line changenamespace Jiminny\Component\ES\Listeners;
namespace Jiminny\Component\ES\Listeners;...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-4a1 github.com/jiminny/app/pull/12082/changes#diff-4a1e0ff3a25c28615f1bf22dc2d6b441b1d1926a2befa1226494a7aed1aeef53...
|
55702
|
|
55650
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said what about non test files
You said
what about non test files
Listen
Show thinking
Gemini said
Gemini said
Since we don't have the actual
SelectionList.php
file, we have to reverse-engineer its implementation based on the test behaviors. Moving from a constructor-based, fixed-property model to a dynamic, mutable model introduces several potential pitfalls in the implementation file itself.
Here are the specific bottlenecks and issues to watch out for in the
SelectionList.php
source code:
1. The count() Method Implementation (Performance Trap)
1. The
count()
Method Implementation (Performance Trap)
The Risk:
If
count()
is implemented by calling
$this->getAllIds()
and counting the resulting merged array (e.g.,
return count($this->getAllIds());
), it will be highly inefficient. It would allocate a brand-new array in memory just to determine its length.
The Fix:
count()
should mathematically sum the sizes of the underlying group arrays without merging them:
return array_sum(array_map('count', $this->entityLists));
2. Loss of Immutability (State Management)
2. Loss of Immutability (State Management)
The Risk:
The original code passed all data via the constructor, implying the DTO (Data Transfer Object) was immutable once created. The refactored version introduces mutability (
addEntityList()
,
setChunkSize()
,
markStale()
). DTOs are typically passed between architectural layers; if one layer modifies the chunk size or adds entities while another is processing it, it can cause unpredictable side effects or race conditions in your queue processor.
The Fix:...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55650
|
|
55651
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55651
|
|
55652
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55652
|
|
55653
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55653
|
|
55654
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
Edit
You said what about non test files
You said
what about non test files
Listen
Show thinking
Gemini said
Gemini said
Since we don't have the actual
SelectionList.php
file, we have to reverse-engineer its implementation based on the test behaviors. Moving from a constructor-based, fixed-property model to a dynamic, mutable model introduces several potential pitfalls in the implementation file itself.
Here are the specific bottlenecks and issues to watch out for in the
SelectionList.php
source code:
1. The count() Method Implementation (Performance Trap)
1. The
count()
Method Implementation (Performance Trap)
The Risk:
If
count()
is implemented by calling
$this->getAllIds()
and counting the resulting merged array (e.g.,
return count($this->getAllIds());
), it will be highly inefficient. It would allocate a brand-new array in memory just to determine its length.
The Fix:
count()
should mathematically sum the sizes of the underlying group arrays without merging them:
return array_sum(array_map('count', $this->entityLists));
2. Loss of Immutability (State Management)
2. Loss of Immutability (State Management)
The Risk:
The original code passed all data via the constructor, implying the DTO (Data Transfer Object) was immutable once created. The refactored version introduces mutability (
addEntityList()
,
setChunkSize()
,
markStale()
). DTOs are typically passed between architectural layers; if one layer modifies the chunk size or adds entities while another is processing it, it can cause unpredictable side effects or race conditions in your queue processor.
The Fix:
Ensure that this object is not shared across asynchronous contexts, or consider returning cloned instances when mutating state if strict immutability is required.
3. Enum Array Key Violations (PHP Quirks)
3. Enum Array Key Violations (PHP Quirks)
The Risk:
The tests show
QueuePriorityEnum::HIGH
being passed to methods. If the underlying code attempts to use this Enum directly as an array key (e.g.,
$this->lists[$priority] = $ids;
), PHP will throw an error because objects (including Enums) cannot be used as standard array keys.
The Fix:
The implementation must map the Enum to its underlying scalar value (e.g.,
$this->lists[$priority->value]
) or use PHP's
WeakMap
or
SplObjectStorage
, though standard arrays using
$priority->value
are much faster for simple lists.
4. Uninitialized State Defaults (Logic Errors)
4. Uninitialized State Defaults (Logic Errors)
The Risk:
The tests instantiate
new SelectionList()
without arguments, then later call
$selectionList->setChunkSize(100)
. If
isFullChunk()
is called
before
setChunkSize()
is invoked, what happens?
The Fix:
Ensure
SelectionList
has a sensible default for chunk size (e.g.,
private int $chunkSize = 0;
or a system default) to prevent uninitialized property errors or division-by-zero scenarios if your logic ever relies on chunk math.
5. getAllIds() Repeated Execution (CPU Overhead)
5.
getAllIds()
Repeated Execution (CPU Overhead)
The Risk:
If
getAllIds()
dynamically merges arrays (
array_merge(...$this->lists)
) every time it is invoked, and your ES processor calls this method inside a loop, it will spike CPU usage.
The Fix:
If the list is read frequently, the implementation should either cache the merged result and invalidate that cache when
addEntityList()
is called, or the caller should assign the result to a local variable before iterating.
Would you like me to draft an optimal, memory-efficient skeleton for the
SelectionList.php
implementation that avoids these specific pitfalls?
Good response...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55654
|
|
55655
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said what about non test files
You said
what about non test files
Listen
Show thinking
Gemini said
Gemini said
Since we don't have the actual
SelectionList.php
file, we have to reverse-engineer its implementation based on the test behaviors. Moving from a constructor-based, fixed-property model to a dynamic, mutable model introduces several potential pitfalls in the implementation file itself.
Here are the specific bottlenecks and issues to watch out for in the
SelectionList.php
source code:
1. The count() Method Implementation (Performance Trap)
1. The
count()
Method Implementation (Performance Trap)
The Risk:
If
count()
is implemented by calling
$this->getAllIds()
and counting the resulting merged array (e.g.,
return count($this->getAllIds());
), it will be highly inefficient. It would allocate a brand-new array in memory just to determine its length.
The Fix:
count()
should mathematically sum the sizes of the underlying group arrays without merging them:
return array_sum(array_map('count', $this->entityLists));
2. Loss of Immutability (State Management)
2. Loss of Immutability (State Management)
The Risk:
The original code passed all data via the constructor, implying the DTO (Data Transfer Object) was immutable once created. The refactored version introduces mutability (
addEntityList()
,
setChunkSize()
,
markStale()
). DTOs are typically passed between architectural layers; if one layer modifies the chunk size or adds entities while another is processing it, it can cause unpredictable side effects or race conditions in your queue processor.
The Fix:
Ensure that this object is not shared across asynchronous contexts, or consider returning cloned instances when mutating state if strict immutability is required.
3. Enum Array Key Violations (PHP Quirks)
3. Enum Array Key Violations (PHP Quirks)
The Risk:
The tests show
QueuePriorityEnum::HIGH
being passed to methods. If the underlying code attempts to use this Enum directly as an array key (e.g.,
$this->lists[$priority] = $ids;
), PHP will throw an error because objects (including Enums) cannot be used as standard array keys.
The Fix:
The implementation must map the Enum to its underlying scalar value (e.g.,
$this->lists[$priority->value]
) or use PHP's
WeakMap
or
SplObjectStorage
, though standard arrays using
$priority->value
are much faster for simple lists.
4. Uninitialized State Defaults (Logic Errors)
4. Uninitialized State Defaults (Logic Errors)
The Risk:
The tests instantiate
new SelectionList()
without arguments, then later call
$selectionList->setChunkSize(100)
. If
isFullChunk()
is called
before
setChunkSize()
is invoked, what happens?
The Fix:
Ensure
SelectionList
has a sensible default for chunk size (e.g.,
private int $chunkSize = 0;
or a system default) to prevent uninitialized property errors or division-by-zero scenarios if your logic ever relies on chunk math.
5. getAllIds() Repeated Execution (CPU Overhead)
5.
getAllIds()
Repeated Execution (CPU Overhead)
The Risk:
If
getAllIds()
dynamically merges arrays (
array_merge(...$this->lists)
) every time it is invoked, and your ES processor calls this method inside a loop, it will spike CPU usage.
The Fix:
If the list is read frequently, the implementation should either cache the merged result and invalidate that cache when
addEntityList()
is called, or the caller should assign the result to a local variable before iterating.
Would you like me to draft an optimal, memory-efficient skeleton for the
SelectionList.php
implementation that avoids these specific pitfalls?
Good response
Bad response
Share & export
Copy
Show more options...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55655
|
|
55661
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Gemini said
Gemini said
Here is a precise and concise summary of the code selection, which refactors
SelectionListTest
to accommodate a shift from fixed high/normal priority properties to a more dynamic, enum-driven priority group architecture.
Refactored Priority Handling
Refactored Priority Handling
Dynamic Priority Groups:
Replaces fixed high/normal array parameters with the
QueuePriorityEnum
(
HIGH
,
NORMAL
,
LOW
), allowing entity lists to be managed dynamically by group types.
Merged Registrations:
Validates that multiple
addEntityList()
calls targeting the same priority level properly merge their data instead of overwriting it.
Group Querying:
Tests new methods
getEntityListItemsByGroup()
and
getEntityListGroups()
to retrieve group-specific IDs or an array of active group names (e.g.,
['high', 'normal']
).
Updated State and Count Validations
Updated State and Count Validations
Instantiation:
Tests now instantiate
SelectionList
without initial constructor arguments, using setters and action methods instead.
Empty States:
Updates assertions (
isEmpty()
) to verify state based on whether any entities have been dynamically added, rather than checking pre-defined constructor arrays.
Stale Flags:
Changes from passing a
stale
boolean via the constructor to explicitly triggering a mutable
$selectionList->markStale()
method.
Chunk Capacity Behavior
Chunk Capacity Behavior
Flexible Chunk Sizes:
Introduces a
setChunkSize()
method, replacing the constructor-configured chunk boundary.
Capacity Assertions:
Retains and updates the logic for
isFullChunk()
, ensuring it accurately evaluates capacity when elements span multiple dynamic priority groups or when the chunk limit is altered mid-lifecycle.
Good response
Bad response
Share & export
Copy
Show more options
Copy prompt
You said Are there any issues or potential bolltenecks
You said
Are there any issues or potential bolltenecks
Listen
Show thinking
Gemini said
Gemini said
Based on the test diff you provided, we can infer a few potential issues and architectural bottlenecks in the underlying
SelectionList
implementation. While the tests are well-structured, the behavior they describe hints at a few areas you might want to keep an eye on, particularly for an Elasticsearch (ES) processor queue.
Here are the potential issues and bottlenecks to consider:
1. Array Merging Overhead (Performance)
1. Array Merging Overhead (Performance)
The
testAddEntityListMergesMultipleCallsForSamePriority
test shows that calling
addEntityList()
multiple times for the same priority group merges the arrays.
The Bottleneck:
If the underlying implementation uses
array_merge()
inside a loop to append these IDs, it creates a new array in memory every time it's called. If you are processing thousands of entities and appending them in small batches, this can lead to memory bloat and CPU degradation due to constant array reallocation.
Mitigation:
Ensure the implementation handles batch appends efficiently (e.g., using the spread operator
[...$existing, ...$new]
or appending via references) or groups the arrays before doing a single merge.
2. Lack of Uniqueness/Deduplication (Logic)
2. Lack of Uniqueness/Deduplication (Logic)
None of the tests explicitly verify what happens if duplicate IDs are added. For example, if you call
addEntityList(QueuePriorityEnum::HIGH, [1, 2])
and then
addEntityList(QueuePriorityEnum::HIGH, [2, 3])
.
The Issue:
Does the getter return
[1, 2, 2, 3]
or
[1, 2, 3]
? If duplicates are not filtered out, your ES processor might end up indexing the same entity multiple times in a single chunk, which wastes I/O and processor cycles.
Mitigation:
The underlying class should ideally enforce uniqueness (e.g., using
array_unique()
before resolving the chunk, or using ID keys as associative array keys to naturally prevent duplicates).
3. Memory Consumption with Large Chunks (Scalability)
3. Memory Consumption with Large Chunks (Scalability)
The tests introduce
setChunkSize()
. If this chunk size is configured to be very large (e.g., tens of thousands of IDs to bulk-send to Elasticsearch).
The Bottleneck:
Holding massive arrays of IDs in memory across different priority groups can quickly hit PHP memory limits, especially if this
SelectionList
object is kept alive during a long-running daemon or cron job.
Mitigation:
Monitor memory usage if
CHUNK_SIZE
grows, and ensure the
SelectionList
object is properly destroyed (or garbage collected) after the chunk is processed to free up memory.
4. Type Juggling
4. Type Juggling
In the original code, string IDs were used (
['p1', 'p2']
). In the refactored tests, integer IDs are used (
[1, 2]
).
The Issue:
If your system handles UUIDs (strings) vs. relational DB auto-increments (integers), make sure the underlying implementation uses strict comparisons (
===
) where necessary, especially if deduplicating arrays, to prevent PHP from juggling
'1'
and
1
improperly.
Would you like me to draft a quick test case to check how the class currently handles duplicate IDs, or is deduplication intentionally handled further down your pipeline?
Good response
Bad response
Share & export
Copy
Show more options...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55661
|
|
55662
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
Edit
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Show thinking
Gemini said
Gemini said
Here is a summary of the changes made to the
SelectionListTest
file, reflecting a significant refactoring of how the
SelectionList
class manages state and priorities.
Object Instantiation and State Management
Object Instantiation and State Management
Removed Constructor Injection:
The tests no longer pass arrays, chunk sizes, or boolean flags directly into the
SelectionList
constructor. Objects are now instantiated empty (
new SelectionList()
).
Introduced
addEntityList()
:
Added tests for populating the selection list dynamically using
addEntityList()
, which pairs a
QueuePriorityEnum
(e.g.,
HIGH
,
NORMAL
,
LOW
) with an array of IDs.
Merged Priorities:
Added tests confirming that invoking
addEntityList()
multiple times for the same priority group successfully merges the ID arrays.
Updated Data Retrieval
Updated Data Retrieval
Generic Group Getters:
Removed legacy priority-specific getters (e.g.,
getHighPriorityEntityIds()
). These are replaced by tests for
getEntityListItemsByGroup()
, which fetches IDs based on the provided enum.
Unknown Group Handling:
Added validation that
getEntityListItemsByGroup()
safely returns an empty array when querying an unpopulated priority group.
Group Listing:
Added
testGetEntityListGroups()
to verify the retrieval of all active priority group keys (e.g.,
['high', 'normal']
).
Aggregated Fetching:
Maintained and updated tests for
getAllIds()
to ensure it correctly flattens IDs across all newly structured priority groups.
Chunking and Flags
Chunking and Flags
Dynamic Chunk Sizing:
Introduced a
CHUNK_SIZE
constant and replaced constructor-based chunk sizing with a new
setChunkSize()
method.
Chunk Validation:
Expanded
isFullChunk()
tests to verify behavior when exactly at chunk capacity, below capacity, when capacities are split across multiple priority groups, and when the chunk size is changed dynamically.
Stale Status:
Replaced the constructor-injected stale flag with a new
markStale()
method, testing that
isStaleChunkSelected()
correctly toggles from false to true.
Utility Methods:
Updated tests for
count()
and
isEmpty()
to align with the new
addEntityList
structure rather than constructor arrays.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Enter a prompt for Gemini
encrypted
Enter a prompt for Gemini
encrypted
Open upload file menu
Tools
Open mode picker
Pro
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality (1)
Security and quality
(
1
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY-20808 low priority indexing queue #12082 Edit title
JY-20808 low priority indexing queue
#
12082
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
Lines changed: 432 additions & 159 deletions
Conversation (5)
Conversation
(
5
)
Commits (36)
Commits
(
36
)
Checks (3)
Checks
(
3
)
Files changed (18)
Files changed
(
18
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
All commits
All commits
0 of 18 files viewed
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
app
Component/ES
Listeners
UpdateMultipleTargetsListener.php
UpdateMultipleTargetsListener.php
UpdateSingleTargetListener.php...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55662
|
|
55663
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Copy prompt
Edit
You said I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>JY-20808 low priority indexing queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -5,84 +5,140 @@5namespace Tests\Unit\Component\ES\Processor\DTOs;5namespace Tests\Unit\Component\ES\Processor\DTOs;667use Jiminny\Component\ES\Processor\DTOs\SelectionList;7use Jiminny\Component\ES\Processor\DTOs\SelectionList;8+use Jiminny\Component\ES\QueuePriorityEnum;8use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\Attributes\CoversClass;9use PHPUnit\Framework\TestCase;10use PHPUnit\Framework\TestCase;101111#[CoversClass(SelectionList::class)]12#[CoversClass(SelectionList::class)]12final class SelectionListTest extends TestCase13final class SelectionListTest extends TestCase13{14{14-public function testGetEntityIds(): void15+private const int CHUNK_SIZE = 100;16+17+public function testAddEntityListAndGetAllIds(): void18+ {19+$selectionList = new SelectionList();20+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);21+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);22+23+$this->assertSame([1, 2, 3, 4], $selectionList->getAllIds());24+ }25+26+public function testAddEntityListMergesMultipleCallsForSamePriority(): void27+ {28+$selectionList = new SelectionList();29+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);30+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [3, 4]);31+32+$this->assertSame([1, 2, 3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));33+ }34+35+public function testGetEntityListItemsByGroup(): void36+ {37+$selectionList = new SelectionList();38+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1, 2]);39+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [3, 4]);40+$selectionList->addEntityList(QueuePriorityEnum::LOW, [5, 6]);41+42+$this->assertSame([1, 2], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));43+$this->assertSame([3, 4], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::NORMAL));44+$this->assertSame([5, 6], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::LOW));45+ }46+47+public function testGetEntityListItemsByGroupReturnsEmptyArrayForUnknownGroup(): void15 {48 {16-$priority = ['p1', 'p2'];49+$selectionList = new SelectionList();17-$normal = ['n1', 'n2'];50+18-$selectionList = new SelectionList($priority, $normal);51+$this->assertSame([], $selectionList->getEntityListItemsByGroup(QueuePriorityEnum::HIGH));52+ }195320-$this->assertSame($priority, $selectionList->getHighPriorityEntityIds());54+public function testGetEntityListGroups(): void21-$this->assertSame($normal, $selectionList->getNormalPriorityEntityId());55+ {56+$selectionList = new SelectionList();57+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);58+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2]);225923-$this->assertSame(['p1', 'p2', 'n1', 'n2'], $selectionList->getAllIds());60+$this->assertSame(['high', 'normal'], $selectionList->getEntityListGroups());24 }61 }256226public function testCountReturnsTotalCount(): void63public function testCountReturnsTotalCount(): void27 {64 {28-$selectionList = new SelectionList(['p1'], ['n1', 'n2']);65+$selectionList = new SelectionList();66+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);67+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [2, 3]);68+29$this->assertSame(3, $selectionList->count());69$this->assertSame(3, $selectionList->count());30 }70 }317132-public function testIsEmptyReturnsTrueWhenBothListsAreEmpty(): void72+public function testIsEmptyReturnsTrueWhenNoEntitiesAdded(): void33 {73 {34-$selectionList = new SelectionList([], []);74+$selectionList = new SelectionList();75+35$this->assertTrue($selectionList->isEmpty());76$this->assertTrue($selectionList->isEmpty());36 }77 }377838-public function testIsEmptyReturnsFalseWhenAnyListIsNotEmpty(): void79+public function testIsEmptyReturnsFalseWhenEntitiesAdded(): void39 {80 {40-$selectionList = new SelectionList(['p1'], []);81+$selectionList = new SelectionList();41-$this->assertFalse($selectionList->isEmpty());82+$selectionList->addEntityList(QueuePriorityEnum::HIGH, [1]);428343-$selectionList2 = new SelectionList([], ['n1']);84+$this->assertFalse($selectionList->isEmpty());44-$this->assertFalse($selectionList2->isEmpty());45 }85 }468647-public function testIsStaleChunkSelected(): void87+public function testMarkStaleAndIsStaleChunkSelected(): void48 {88 {49-$selectionList = new SelectionList([], [], true);89+$selectionList = new SelectionList();50-$this->assertTrue($selectionList->isStaleChunkSelected());90+91+$this->assertFalse($selectionList->isStaleChunkSelected());92+93+$selectionList->markStale();519452-$selectionList2 = new SelectionList([], [], false);95+$this->assertTrue($selectionList->isStaleChunkSelected());53-$this->assertFalse($selectionList2->isStaleChunkSelected());54 }96 }559756public function testIsFullChunk(): void98public function testIsFullChunk(): void57 {99 {58-$chunkSize = 200;100+$selectionList = new SelectionList();59-101+$selectionList->setChunkSize(self::CHUNK_SIZE);60-$fullList = range(1, $chunkSize);102+61-$selectionList = new SelectionList(103+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, self::CHUNK_SIZE));62- priorityEntityIdList: $fullList,104+63- normalEntityIdList: [],105+$this->assertTrue($selectionList->isFullChunk());64- chunkSize: $chunkSize,106+ }65- );107+108+public function testIsNotFullChunkWhenBelowChunkSize(): void109+ {110+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);111+112+$selectionList = new SelectionList();113+$selectionList->setChunkSize(self::CHUNK_SIZE);114+115+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range(1, $halfRange));116+117+$this->assertFalse($selectionList->isFullChunk());118+ }119+120+public function testIsFullChunkAcrossMultiplePriorityGroups(): void121+ {122+$halfRange = (int) ceil(self::CHUNK_SIZE / 2);123+124+$selectionList = new SelectionList();125+$selectionList->setChunkSize(self::CHUNK_SIZE);126+127+$selectionList->addEntityList(QueuePriorityEnum::HIGH, range(1, $halfRange));128+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, range($halfRange + 1, self::CHUNK_SIZE));129+130+$this->assertTrue($selectionList->isFullChunk());131+ }132+133+public function testSetChunkSizeAffectsIsFullChunk(): void134+ {135+$selectionList = new SelectionList();136+$selectionList->addEntityList(QueuePriorityEnum::NORMAL, [1, 2, 3]);137+138+$selectionList->setChunkSize(3);66$this->assertTrue($selectionList->isFullChunk());139$this->assertTrue($selectionList->isFullChunk());6714068-$halfRange = (int) ceil($chunkSize / 2);141+$selectionList->setChunkSize(50);69-142+$this->assertFalse($selectionList->isFullChunk());70-$notFullList = range(1, $halfRange);71-$selectionList2 = new SelectionList(72- priorityEntityIdList: $notFullList,73- normalEntityIdList: [],74- chunkSize: $chunkSize,75- );76-$this->assertFalse($selectionList2->isFullChunk());77-78-$priorityRange = range(1, $halfRange);79-$normalRange = range($halfRange + 1, $chunkSize);80-81-$selectionList3 = new SelectionList(82- priorityEntityIdList: $priorityRange,83- normalEntityIdList: $normalRange,84- chunkSize: $chunkSize,85- );86-$this->assertTrue($selectionList3->isFullChunk());87 }143 }88}144}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Listen
Show thinking
Gemini said
Gemini said
Here is a summary of the changes made to the
SelectionListTest
file, reflecting a significant refactoring of how the
SelectionList
class manages state and priorities.
Object Instantiation and State Management
Object Instantiation and State Management
Removed Constructor Injection:
The tests no longer pass arrays, chunk sizes, or boolean flags directly into the
SelectionList
constructor. Objects are now instantiated empty (
new SelectionList()
).
Introduced
addEntityList()
:
Added tests for populating the selection list dynamically using
addEntityList()
, which pairs a
QueuePriorityEnum
(e.g.,
HIGH
,
NORMAL
,
LOW
) with an array of IDs.
Merged Priorities:
Added tests confirming that invoking
addEntityList()
multiple times for the same priority group successfully merges the ID arrays.
Updated Data Retrieval
Updated Data Retrieval
Generic Group Getters:
Removed legacy priority-specific getters (e.g.,
getHighPriorityEntityIds()
). These are replaced by tests for
getEntityListItemsByGroup()
, which fetches IDs based on the provided enum.
Unknown Group Handling:
Added validation that
getEntityListItemsByGroup()
safely returns an empty array when querying an unpopulated priority group.
Group Listing:
Added
testGetEntityListGroups()
to verify the retrieval of all active priority group keys (e.g.,
['high', 'normal']
).
Aggregated Fetching:
Maintained and updated tests for
getAllIds()
to ensure it correctly flattens IDs across all newly structured priority groups.
Chunking and Flags
Chunking and Flags
Dynamic Chunk Sizing:
Introduced a
CHUNK_SIZE
constant and replaced constructor-based chunk sizing with a new
setChunkSize()
method.
Chunk Validation:
Expanded
isFullChunk()
tests to verify behavior when exactly at chunk capacity, below capacity, when capacities are split across multiple priority groups, and when the chunk size is changed dynamically.
Stale Status:
Replaced the constructor-injected stale flag with a new
markStale()
method, testing that
isStaleChunkSelected()
correctly toggles from false to true.
Utility Methods:
Updated tests for
count()
and
isEmpty()
to align with the new
addEntityList
structure rather than constructor arrays.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Enter a prompt for Gemini
encrypted
Enter a prompt for Gemini
encrypted
Open upload file menu
Tools
Open mode picker
Pro
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality (1)
Security and quality
(
1
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY-20808 low priority indexing queue #12082 Edit title
JY-20808 low priority indexing queue
#
12082
Edit title
Preview...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55663
|
|
55676
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55676
|
|
55677
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions....
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55677
|
|
55678
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
PRO
PRO
Conversation with Gemini
Conversation with Gemini
Hi Lukas
Where should we start?
Where should we start?
🖼️ Create image, button, tap to use tool
🖼️ Create image
🎸 Create music, button, tap to use tool
🎸 Create music
Write anything, button, tap to use tool
Write anything
Boost my day, button, tap to use tool
Boost my day
Help me learn, button, tap to use tool
Help me learn
Mak
Mak
Open upload file menu
Tools
Open mode picker
Pro
Send message
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality (1)
Security and quality
(
1
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY-20808 low priority indexing queue #12082 Edit title
JY-20808 low priority indexing queue
#
12082
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
Lines changed: 432 additions & 159 deletions
Conversation (5)
Conversation
(
5
)
Commits (36)
Commits
(
36
)
Checks (3)
Checks
(
3
)
Files changed (18)
Files changed
(
18
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
JY-20808 low priority indexing queue
JY-20808 low priority indexing queue
#
12082
All commits
All commits
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
0 of 18 files viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
app
Component/ES
Listeners
UpdateMultipleTargetsListener.php
UpdateMultipleTargetsListener.php
UpdateSingleTargetListener.php
UpdateSingleTargetListener.php
Processor
Actions
UpsertDocumentsAction.php
UpsertDocumentsAction.php
DTOs
SelectionList.php
SelectionList.php
Traits
SelectEntityListTrait.php
SelectEntityListTrait.php
TargetEntitiesSelector.php
TargetEntitiesSelector.php
AsyncUpdateElasticSearch.php
AsyncUpdateElasticSearch.php
QueuePriorityEnum.php
QueuePriorityEnum.php
UpdateProcessManager.php
UpdateProcessManager.php
Console/Commands/Elasticsearch
AsyncUpdateEsEntities.php
AsyncUpdateEsEntities.php
ResetAsyncElasticSearchCommand.php
ResetAsyncElasticSearchCommand.php
tests/Unit/Component/ES
Collapse file
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
Copy file name to clipboard
Expand all lines: app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
Lines changed: 2 additions & 1 deletion
Not Viewed
Viewed
Comment on this file
More options
Original file line number
Original file line
Diff line number
Diff line change
@@ -5,6 +5,7 @@
5...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55678
|
|
55679
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
PRO
PRO
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
Edit
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Enter a prompt for Gemini
encrypted
Enter a prompt for Gemini
encrypted
Open upload file menu
Tools
Open mode picker
Pro
Stop response
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Gemini is typing
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality (1)
Security and quality
(
1
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY-20808 low priority indexing queue #12082 Edit title
JY-20808 low priority indexing queue
#
12082
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
Lines changed: 432 additions & 159 deletions
Conversation (5)
Conversation
(
5
)
Commits (36)
Commits
(
36
)
Checks (3)
Checks
(
3
)
Files changed (18)
Files changed
(
18
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
JY-20808 low priority indexing queue
JY-20808 low priority indexing queue
#
12082
All commits
All commits
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
0 of 18 files viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
app
Component/ES
Listeners
UpdateMultipleTargetsListener.php
UpdateMultipleTargetsListener.php
UpdateSingleTargetListener.php
UpdateSingleTargetListener.php
Processor
Actions
UpsertDocumentsAction.php
UpsertDocumentsAction.php
DTOs
SelectionList.php
SelectionList.php
Traits
SelectEntityListTrait.php
SelectEntityListTrait.php
TargetEntitiesSelector.php
TargetEntitiesSelector.php
AsyncUpdateElasticSearch.php
AsyncUpdateElasticSearch.php
QueuePriorityEnum.php
QueuePriorityEnum.php
UpdateProcessManager.php
UpdateProcessManager.php
Console/Commands/Elasticsearch
AsyncUpdateEsEntities.php
AsyncUpdateEsEntities.php
ResetAsyncElasticSearchCommand.php
ResetAsyncElasticSearchCommand.php
tests/Unit/Component/ES
Collapse file
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
Copy file name to clipboard
Expand all lines: app/Component/ES/Listeners/UpdateMultipleTargetsListener.php
Lines changed: 2 additions & 1 deletion
Not Viewed
Viewed
Comment on this file
More options
Original file line number
Original file line
Diff line number
Diff line change
@@ -5,6 +5,7 @@
5
namespace
Jiminny
\
Component
\
ES
\
Listeners
;
5
namespace
Jiminny
\
Component
\
ES
\
Listeners
;
6
6
7
use
Jiminny
\
Component
\
ES
\
AsyncUpdateElasticSearch
;
7
use
Jiminny
\
Component
\
ES
\
AsyncUpdateElasticSearch
;
8
+
use...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55679
|
|
55680
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
Edit
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff,
likely from a version control system given the red and green highlights.
I would love to help you review the changes,
spot bottlenecks,
and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny,
illegible pixels.
Because of this,
I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively,
Enter a prompt for Gemini
encrypted
Enter a prompt for Gemini
encrypted
Open upload file menu
Tools
Open mode picker
Pro
Stop response
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality (1)
Security and quality
(
1
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY-20808 low priority indexing queue #12082 Edit title
JY-20808 low priority indexing queue
#
12082
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
Lines changed: 432 additions & 159 deletions
Conversation (5)
Conversation
(
5
)
Commits (36)
Commits
(
36
)
Checks (3)
Checks
(
3
)
Files changed (18)
Files changed
(
18
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
JY-20808 low priority indexing queue
JY-20808 low priority indexing queue
#
12082
All commits
All commits
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
0 of 18 files viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
app
Component/ES
Listeners
UpdateMultipleTargetsListener.php
UpdateMultipleTargetsListener.php
UpdateSingleTargetListener.php
UpdateSingleTargetListener.php
Processor
Actions
UpsertDocumentsAction.php
UpsertDocumentsAction.php
DTOs
SelectionList.php
SelectionList.php
Traits
SelectEntityListTrait.php
SelectEntityListTrait.php
TargetEntitiesSelector.php
TargetEntitiesSelector.php
AsyncUpdateElasticSearch.php
AsyncUpdateElasticSearch.php
QueuePriorityEnum.php
QueuePriorityEnum.php
UpdateProcessManager.php...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55680
|
|
55681
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
Edit
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Redo
Share & export
Copy
Show more options
Enter a prompt for Gemini
encrypted
Enter a prompt for Gemini
encrypted
Open upload file menu
Tools
Open mode picker
Pro
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality (1)
Security and quality
(
1
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY-20808 low priority indexing queue #12082 Edit title
JY-20808 low priority indexing queue
#
12082
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
Lines changed: 432 additions & 159 deletions
Conversation (5)
Conversation
(
5
)
Commits (36)
Commits
(
36
)
Checks (3)
Checks
(
3
)
Files changed (18)
Files changed
(
18
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
JY-20808 low priority indexing queue
JY-20808 low priority indexing queue
#
12082
All commits
All commits
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
0 of 18 files viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
app
Component/ES
Listeners
UpdateMultipleTargetsListener.php
UpdateMultipleTargetsListener.php
UpdateSingleTargetListener.php
UpdateSingleTargetListener.php
Processor
Actions
UpsertDocumentsAction.php
UpsertDocumentsAction.php
DTOs...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55681
|
|
55682
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20725 add HS rate limit handling on activities rematching by LakyLak · Pull Request #12066 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app
Close tab
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close
WORK, Google Account: [EMAIL]
Main menu
New chat
Gemini
New Chat
Open menu for conversation actions.
Conversation with Gemini
Conversation with Gemini
Show the uploaded image in a lightbox
Copy prompt
Edit
You said Make a PR of changes Find potential issues and bottlenecks
You said
Make a PR of changes Find potential issues and bottlenecks
Listen
Show thinking
Gemini said
Gemini said
I can see you've uploaded a screenshot of a code diff, likely from a version control system given the red and green highlights. I would love to help you review the changes, spot bottlenecks, and help you draft a Pull Request.
However,
the image resolution is far too low for me to read.
It appears to be a completely zoomed-out view where the code has been reduced to tiny, illegible pixels. Because of this, I cannot extract any text or logic to analyze.
How You Can Share the Code
How You Can Share the Code
To help me review your code effectively, please try one of the following:
Paste the text directly:
Copy the specific code blocks (both the old and new versions if possible) and paste them directly into our chat.
Upload the files:
If the changes are extensive, you can upload the raw text or code files.
Take high-resolution screenshots:
If you must use images, take closer, readable screenshots of the specific functions or methods you want me to review.
Once you provide the readable code, I can dive right in to identify potential bugs, assess performance bottlenecks, and help you summarize the changes for your PR!
Good response
Bad response
Redo
Share & export
Copy
Show more options
Enter a prompt for Gemini
encrypted
Enter a prompt for Gemini
encrypted
Open upload file menu
Tools
Open mode picker
Pro
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality (1)
Security and quality
(
1
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
JY-20808 low priority indexing queue #12082 Edit title
JY-20808 low priority indexing queue
#
12082
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
Vasil-Jiminny
Vasil-Jiminny
wants to merge 36 commits into
master
master
from
JY-20808-low-priority-indexing-queue
JY-20808-low-priority-indexing-queue
Copy head branch name to clipboard
Lines changed: 432 additions & 159 deletions
Conversation (5)
Conversation
(
5
)
Commits (36)
Commits
(
36
)
Checks (3)
Checks
(
3
)
Files changed (18)
Files changed
(
18
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
All commits
All commits
0 of 18 files viewed
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
app
Component/ES
Listeners
UpdateMultipleTargetsListener.php
UpdateMultipleTargetsListener.php
UpdateSingleTargetListener.php
UpdateSingleTargetListener.php
Processor
Actions
UpsertDocumentsAction.php
UpsertDocumentsAction.php
DTOs
SelectionList.php
SelectionList.php
Traits
SelectEntityListTrait.php
SelectEntityListTrait.php
TargetEntitiesSelector.php
TargetEntitiesSelector.php
AsyncUpdateElasticSearch.php
AsyncUpdateElasticSearch.php...
|
Firefox
|
JY-20808 low priority indexing queue by Vasil-Jimi JY-20808 low priority indexing queue by Vasil-Jiminny · Pull Request #12082 · jiminny/app — Work...
|
github.com/jiminny/app/pull/12082/changes#diff-49a github.com/jiminny/app/pull/12082/changes#diff-49a79d99c97caf7fc27ecb1dbacab5a29d871822caa0db0c71e94957c28c87af...
|
55682
|